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STRICT ERGODICITY AND TRANSFORMATION 
OF THE TORUS.* 


By H. FURSTENBERG. 


Introduction. If 7 is a measure preserving transformation of a prob- 


ability space 2 with measure p, the ergodic theorem assures the existence 
N-1 

almost everywhere with respect to » of the average lim NS f(T"w), where 
N> n=0 


f is an integrable function. Can this statement be improved in case Q 


is a compact topological space, T a suitable homeomorphism of Q with 
itself, and f a continuous function on Q? In particular, can convergence 
almost everywhere be replaced by convergence everywhere? In what follows 
we shall examine this question for the case that © is an r-dimensional torus. 
When r—1, i.e. Q is a circle, the answer is in the affirmative and the 
averages in question always exist (a fact implicit in the results of Denjoy 
[1] and van Kampen [5]). For 7 > 1, however, further restrictions must be 
imposed on the transformation 7 and part of our objective will be to exhibit 
a class of T for which this sharpened form of the ergodic theorem holds. 

The question we are considering is closely tied up with that of the 
“strict ergodicity ” of a transformation. A transformation 7 of a compact 
Hausdorff space Q is strictly ergodic if it leaves invariant a unique probability 
measure on the borel field of ©. This notion was first introduced in connection 
with the theory of dynamical systems by Kryloff and Bogoliuboff ([6]; cf. 
also [8], [9]). When T is a strictly ergodic transformation, then (Theorem 
1.1) the limits of N23 f(T) necessarily exist for f continuous and all 
w€Q, and moreover, this Tasit is independent of ». In the case of a strictly 
ergodic transformation, these conclusions are in fact a good deal more 
elementary than the usual ergodic theorem. Thus it is quite natural to 
inquire when a transformation of a given space will be strictly ergodic. 

As we will see, an important condition for the validity of some of our 
conclusions is that the transformation T not be homotopic to the identity 
transformation. This implies that the transformation 7’ cannot be embedded 
in a continuous transformation group 7'(¢) and so, in particular, could not 
arise from the consideration of dynamical systems on the torus. The homotopy 
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condition also implies that the transformation is not “uniformly almost 
periodic,” and the sample sequences (n) —f(7T"w) provide us with new 
classes of “constructible” sequences possessing mean values without being 
almost periodic in the sense of Bohr or Besicovitch. 

A related problem is that of determining whether the space Q is “ mini- 
mal” for the transformation 7. A closed invariant set is minimal for T if 
no proper closed subset is invariant under 7’. In the situation we shall study, 
the transformation 7’ will also possess this irreducibility property; that is, 
2 will be a minimal set. It will appear that the hypotheses required for 
minimality and for strict ergodicity are very similar, although not quite 
identical. 


1. Preliminaries. 


1.1. Let Q be a compact metric space, C(Q) its algebra of continuous 
complex valued functions, and 7’ a 1-1 continuous transformation of © onto 
itself. If f is a function on Q we shall denote by 7f the function satisfying 
Tf(w) =f(To). There always exists some probability meause p on the borel 
field of O invariant under 7. This follows from a general fixed point theorem 
regarding transformations of a compact convex set. It can also be seen by 
choosing a countable dense set of — {fx} in C(Q), a point w€ Q, and 


a sequence NV; such that exists for all &. It follows then 


that this limit exists for ‘all fe C i) and this defines a linear functional L 
on C(Q). LZ is non-negative for non-negative function, 1 for the function 1, 
and L(7Tf) —L/(f). It follows from the Riesz-Markoff representation theorem 
that Z defines an invariant measure. 


Definition 1. A continuous transformation T of Q is called strictly 
ergodic if there is a unique probability measure on the borel field of © 
satisfying (A) —=y(T-"A) for every borel set A in Q. 

We remark that this definition, which is that of ([6]), differs slightly 
from the more current usage as in [8] and [9]. In the latter, what we have 
defined is referred to as “unique ergodicity,” and a transformation is strictly 
ergodic only if in addition, the support of the unique invariant measure is all 
of ©. Since the support of an invariant measure is a closed invariant set, 
this means that © is required to be a minimal set. 

Given a probability measure » on ©, then a measure preserving trans- 
formation is ergodic in the usual sense if all borel subsets invariant under 
T have measure either 0 or 1. This is equivalent to saying that no prob- 
ability measure absolutely continuous with respect to » is invariant, other 
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than p itself. It follows that if T is a strictly ergodic transformation on Q, 
it is a fortiori ergodic in the usual sense, relative to the unique invariant 
measure. The set of all probability measures on Q invariant under T form 
a compact convex set, and one may verify that the extremal points of this 
set are just the ergodic measures for 7’, i.e. the measures with respect to 
which 7 is ergodic. Hence to show that a transformation is strictly ergodic, 
it suffices to show that there is a unique ergodic measure for the transformation. 

We shall refer to the triple (Q, 7,) when yp is a probability measure on 
Q invariant under 7’, as a process. If we were to fix a function z() on Q, 
then the functions 7"z(w) would form a stationary stochastic process. When 
T is ergodic or strictly ergodic we shall refer to the process as ergodic or 
strictly ergodic. 

Definition 2. A point »€Q is a generic point for the process 


N-1 
(0, T,p»), if lim N+ S f(T") exists for every f€ C(Q), and coincides with 
n=0 


flo)dulo). 

Note that this definition differs slightly from that given in [2] where 
it is required in addition that w lie in the support of the measure p. 

When a process is ergodic it follows from the ergodic theorem that 
almost every point of Q is generic. For strictly ergodic processes we have 


([9]): 

THEOREM 1.1. The following statements for a process (Q,T,p) are 
equivalent : 

(A) (Q, is strictly ergodic. 

(3) > converges uniformly to p(f). 

(C) Every “point of Q is generic for (Q,T,p). 

Proof. (A) implies (B): If yp is the only probability measure on 2 
invariant under 7’, it follows that the only signed measures invariant under 
T are Ap, A real. Hence the uniform closure in C(Q) of the subspace 


{Tg — 9,9 € C(Q)} has codimension 1. Consequently, for any f€ C(Q) and 
«> 0, we can find a g€ C(Q) with | f—yp»(f) —(Tg—g)|<«. Hence 


|S —Na(f) — < Ne and Tf 


uniformly. 
That (B) implies (C) is clear so it remains to show that (C) implies (A). 
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N-1 
However, if every point of © is generic for (Q,7,), then N-? >> T"f(o) 


N-1 
converges pointwise to w(f), and hence fN* T"f(w)dv(w) >u(f) for any 
n=0 


probability measure v. In particular if v is invariant under 7, then 
4 -1 


v(f) T*f(w)dv(w) so that v(f) —n(f). 


n=0 
1.2. The best known examples of strictly ergodic processes are those 


arising from almost periodic transformation. 


Definition 3. A transformation T of Q is almost periodic (a.p.) if T 
leaves no proper (4% or ©) closed subset of O invariant and if the powers 
T” form an equicontinuous family of transformation with respect to the 
metric on Q. A process (Q, 7,4), with T almost periodic, will be referred to 
as an almost periodic process. 

By the sample paths or sample sequences of a process (Q, 7’, »), we shall 
understand the sequences of the form é(n) =T"f(w) where f€ C(Q). One 
shows easily that the sample paths of an almost periodic process are (uni- 
formly) almost periodic sequences. 


The following result appears in [8]: 
THEOREM 1.2. Every almost periodic transformation is strictly ergodic. 


Proof. The equicontinuity of the 7” implies the equicontinuity of {7"f} 
for f€ C(Q) and hence also the equicontinuity of the averages {N-? > T"f}. 
It follows that some subsequence converges uniformly; say the limit function 
is g. Clearly Tg =g and since T leaves no proper closed subset of © invariant 
it follows that g is a constant. Hence for any probability measure yp, 
N*> »(T"f) > g and if p is invariant, »(f) —g. Hence T is strictly ergodic. 


1.3. To illustrate the general approach to the problem, we begin by 
examining the case in which 2 is the circle, even though the results here 
follow from known results regarding transformations of the circle. 


THEOREM 1.3. If T is any 1-1 continuous transformation of the circle 
such that no power of T has a fixed point, then T is strictly ergodic. 


Proof. Let K denote the circle. The hypothesis that no power of T 
has a fixed point, or equivalently, that 7 leaves no finite subset of K invariant, 
implies that any invariant measure must assign measure 0 to individual 
points, that is, any invariant measure on K is non-atomic. Let w, be a fixed 
point on K and for two points w,,2€ K, let {w1,@2} denote the open arc 


wy 
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from w; to w. taken counterclockwise. Suppose that » and p’ are two invariant 
probability measures on K and let A=3(u-+ yp’). We define a map ® from 
K to a circle K’ by setting ®(w) = e?™K%o}), Since A is non-atomic, ® will 


be a continuous map from K onto K’. 
We note that for any three points o;, w2, 3; in K, 


A({o1, o2}) + A({o2, o3}) =A({o1,03}) (mod 1). 
It follows that 


©(Tw) e2T id({wo; Tw}) e2TilA({wo» Tw}) +A(T Tw})] 


where a=A({wo, Two}). If a were rational we would have (7) = ®(w) 
identically for some n. Then A({wo, =A({wo,}) (mod1) so that 
A({o, T"o}) =0 (mod1) and the interval from » to 7» carries either 
measure 0 or 1. Thus either A({o, =0 or A({T"%o, o}) = 0 and which- 
ever holds for some w and Jw and it follows that every interval of sufficiently 
small length on K carries no A measure. This however is impossible since 
40 and this shows that @ is irrational. 

Thus ® carries A into (normalized) lebesgue measure on K’ and it takes 
T into the rotation of K’ by an irrational angle. Now yp and yp’ are carried 
by ® into two measures on K’ that are invariant under rotation by an irrational 
angle, since » and yp’ are invariant under 7. But clearly the rotations of a 
circle preserving a given measure form a closed group. (If Bn— B and f is con- 
tinuous then f(6+ Bn) 8) and ff(9+ Bn)dV(6)— ff(6 + B)dV(6).) 
Hence p» and p’ are carried into probability measures on K’ invariant under all 
rotations, i.e. into lebesgue measure. Thus yp and p’ have the same images 
under ®, On the other hand, if two intervals of K map onto the same interval 
of K’ under ®, their symmetric difference carries 0 A-measure. A fortiori it 
carries 0 p- and p’-measure, and so both intervals have the same p-measure 
and y’-measure. It follows that the measures p and p»’ are determined by their 
images under ® and since these agree, »—p’. This proves that T is strictly 
ergodic. 

Corotuary. Jf T is any continuous transformation of the circle, then 
lim N23 Tf (w) exists for all continuous functions f and points » on the 
circle, 

Proof. If no power of T has a fixed point then the result follows from 
Theorems 1.1 and 1.3. If some 7™ has a fixed point, let T; =7". It is 


q 
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easily shown then that for every point w, 7," converges to a fixed point of 7’,. 


N-1 
Therefore the averages lim N* >} 7,"f() exist for all wo. Applying this to 
n=0 


N-1 
the functions f,7f,- we conclude that lim N- T"f(w) exists. 
n=0 


In case T leaves no proper closed subset of K invariant, then the support 


of the unique invariant measure A must be all of K. The map ® of Theorem 
1.3 is then a homeomorphism and we can say that the process (K,T,A) is 
equivalent to the process induced by an irrational rotation of the circle. In 
particular the process (K,7,) is almost periodic and the sample paths are 
almost periodic sequences with frequencies ka. 


2. Processes on the torus. 


2.1. The following lemma provides us with an inductive procedure for 


defining strictly ergodic processes. 


LemMA 2.1. Let (%,To,"o.) be a strictly ergodic process, let K be the 
circle, K = {£: | £|=1}, and let m be (normalized) lebesgue measure on K. 
Take 9=Q X K and T: QQ defined by T (wo, = (Towo, 9 (wo) £) where 
g(wo) is a continuous function with |g|—=1. Then T is a strictly ergodic 
transformation if and only if the equation 


(1) g* (wo) = R(T (wo) 


has no solution for k an integer A0 and R(w) a measurable function on 
(Qo, wo). In any case, the product measure po Xm is an invariant measure 
for T. 


Proof. To verify that ». Xm is an invariant measure, set f(w) 


=fi(wo)fe(¢). Then Tf (o) = and 


— fale) dm) (00) duo (00) 


= J, f(w)dp X m(o). 


We claim that in order that T be strictly ergodic on Q it suffices that it be 
ergodic in the ordinary sense with respect to p= X m. To see this define 
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the transformation Sg: 92> by Sg(w, £) = (wo, e?8Z). Clearly Sg pre- 
serves the measure p. It follows easily that if w is a generic point for 
(Q, 7,4) then so is Sg(w) for every B€ [0,1). Now suppose that T is ergodic 
with respect to ». Then almost all points of Q (with respect to w) are generic 
for (Q,7,). Hence for almost all w) (with respect to yo), all points (wo, ¢) 
are generic for (Q,7,). Suppose next that there exists some other ergodic 
measure p’ for 7. Since a T-invariant measure on Q induces a T-invariant 
measure on Q, in a natural way, and since (Q,7'o, 40) is strictly ergodic, p’ 
nust induce the same measure po On Qo. Then for almost all wo in Q) there 
exist points (w,) generic for (0,7, »’). But this is impossible if p’~yp, 
since for almost all wo, all (o,f) are generic for (0,7,4). Thus if 
4= lo X m is ergodic, it is the unique ergodic invariant measure and hence 
the unique invariant measure (cf. the remarks following Definition 1). 

To determine whether y is ergodic, suppose that Tf =f where f € L°(Q, u). 


Since » is a product measure we can write f ~ > cx(wo)f with c(wo) € L?(Qo, pro). 


Tf=f implies > Cx( T oo) g* (wo) = or ( Tow.) = for 


every k. Since 7 is ergodic, f cannot reduce to a function of w, alone and 
hence some ¢;(w) is not 0 for k 40. Since |g|—1 it follows from the 
ergodicity of 7’, that c, vanishes only on a set of measure 0, so that 
R(wo) = Cz(wo) is a solution of (1). 

Conversely, if (1) has a solution, then the function R(w)¢* is invariant 
under 7’ so that » is not ergodic. This completes the proof of the lemma. 

We note that if (1) has a solution R(wo), then | R(w)| is almost every- 
where constant and we may assume that | R(w.)| 1. 


2.2. If @ is a map from a topological space to the circle K then ¢ is 
called essential if it is not homotopic to the constant map, or equivalently 
if @ does not have a single-valued continuous logarithm. When the space is 
the circle then the increase in ;\,log¢(¢) as ¢ describes the circle once is an 
integer referred to as the degree of ¢. Thus ¢ is essential if and only if its 
degree does not vanish. 


Lemma 2.1 provides us with strict ergodicity in case the function g (wo) 
on Q, satisfies a certain condition with respect to T,. In the next lemma we 
find a particular form for (Qo, 7')) and g(wo) so that the condition is satisfied. 
Here Q, is itself the product of a space Q, and the circle, and the points wo 
of Q) may be represented as (1, €), € K. 


LemMA 2.2. Let Qo =Q, X K where Q, 1s a connected metric space, let 
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pi be a probability measure on Q,, Q, and T, a continuous measure preserving 
transformation of Q,. Suppose that To: is gwen by T £) = 
and that T, is ergodic with respect to the measure py XK m—po On Qo. Let 
9(1,¢) be a continuous function on Q, such that g(w1,€) is an essential map 
of K—K for fixed w, and satisfying a uniform Lipschitz condition in ¢: 


| 2) —g (or 0") | <M |. 


Then equation (1), g*(wo) =R(Towo)/R(wo), has no solution for k A0 and 
R(wo) measurable on (Qo, 


Proof. Since Q, is connected, the degree d of the map g: 0. X K>K 
is independent of ,, and, in particular, if the map is essential for one «,, 
is is so for all. The degree of g* will be kd and so g* is again essential in its 
second variable and also satisfies a uniform Lipschitz condition. Thus to 
prove the lemma it suffices to show that g(wo) itself cannot have the form 
R(T owo)/R(wo) with R(wo) a measurable function from © to K. 

Note that if d(.,.) is a metric for Q,, then d((wy’,f), (o1”, ¢’)) 
= is a metric on Q). Now suppose that 
= R(T w.)/R(o.). Since RF is a measurable function it is continuous when 
restricted to some compact subset of Q, with measure > 1—», for arbitrary 
> 0. Hence for a specified 8, 3’ > 0, there exists an > 0 and a subset A 
of with po(A) >1—8 and such that | R(o’) —R(o”)| whenever 
d(wo’, and wy wo’ € A. We set 


A, = {w€ O1: (o,f) € A) 2 1— 


Then A, CQ, and »,(A,) 21— 8. 

If §<1, then A, and hence A, X K will have positive measure. There- 
fore there will exist «/2-neighborhoods in Q, intersecting A, X K in sets of 
positive measure; let V be such a neighborhood. By the ergodicity of 
J) = pa X m, there will exist points in V N(A, X K) such that infinitely many 
of their images under 7’,” again lie in VN(A, XK). Let («,’,&) be such 
a point and suppose that 7'y"*(w1’, € VN(A, X K). 

Now define Sg: as in Lemma 2.1; i.e. Sg(w:,£) = e?7*8£). 
From the definition of To, it is easily checked that each Sg commutes with 7). 
Moreover Sg preserves the metric d(.,.) in Q. By applying Sg to (w’, 2) 
we find that for all d(T "(o1’,£), (o,£)) and both and 
T"*(w,’,f) lie in A, X K. 

From the definition of we find that = (T,%w,’, 
where ; is independent of £. Now since (w,’,£) and T"*(,’,f) lie in A, X K, 


ing 
Let 


wap 
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it follows that the set of £ for which the former is not in A has measure 
< 8, and similarly for the latter. Since the é’s for the two points are fixed 
translates of one another it follows that the set of £ such that either (w,’, £) 
or To"*(w,’,£) is not in A has measure < 284. Putting all this together we 
conclude that but for a set of £ of measure < 2834, 


| —RB(w’,0)| 


To complete the proof of the lemma we observe that | R(T") — R(w) | 
< 8 implies that | -g(Tow)g(o) —1| <8. Now we may write 
g (or, £)) = 9 (Trio, where Since g satisfies 
a uniform Lipschitz condition in ¢ it follows that the h;(6) satisfy uni- 
formly a Lipschitz condition: | h;(0,) —h;(@2)| M|6,—6.|. Since g is 
essential in ¢ we will have hj(6+1)—h’;(6)-+d where d0 is the 


degree of g. Set H,(0) = > A;(0); then H,(0+1) + md and 


| Hy. (01) << Mnx | 0: — 4 |. 

We have then | e?"*#.( —1|<& for all 6 outside a set of 6 of measure 
< 28. Taking & = V2 implies that H;,(6) has to remain within a distance 
4 from an integer, for all 6 outside of a set of measure < 28. But H;,(6) is 
a continuous function, and as @ goes from 0 to 1, H;,(6) has a range of length 
at least n;,d. This implies that in the exceptional set of measure < 283, 
H;,(0) oscillates by 24n,d. But by the Lipschitz condition, in a set of 
measure < 284, H;,(6@) can oscillate by at most 24/n;,64, and this gives a contra- 

d 
4M 

Note that the lemma is true, in particular, when ©, reduces to a single 
point so that Q, is just the circle K. 


diction if 6 < (—~-)*, which proves the lemma. 


2.3 THEoreM 2.1. Let T be a transformation of Q=K" given by 


TE, = e? 
The = 91 (81) &2 


(2) = 93 (E15° 


Where @ is irrational, each g; is a continuous function with | g|—1, each 
+, satisfies a uniform Lipschitz condition in ¢;, and 
is of degree d; in £; where d; 40. Then T is a strictly ergodic transformation 
and the unique invariant measure is the normalized lebesgue measure on K’. 
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Proof. The theorem is valid for r—1 by virtue of Theorem 1.2 or 
Theorem 1.3. Now assume the theorem is valid for Q—K"*. By Lemma 
2.1 it will follow for OK" if the equation 


(wo) R(T wo) (wo) 


has no measurable solution, where 7’, is the induced transformation on K’"! and 
wo € K*1, But the space K** has the form K"* X K and Ty, is induced by T, 
on Kr-? in the manner prescribed in Lemma 2.2. Since the hypotheses of 
Lemma 2.2 are fulfilled, it follows that the equation in question has no 
solution. The theorem therefore follows by induction on r. 

If we denote the invariant probability measure on K* by p, we notice 


that for =u (o(&) (Gs) ), so that for j, and 


are independent variables. Taking 71 we obtain in particular 


if 10. Since this says that the Bohr-Fourier coefficients 
of the sample sequence 7"¢(£;)(w) corresponding to the frequencies lz all 
vanish. Now Theorem 2.1 easily extends to the case where the base process 
(T¢, = e?™'*f,) is replaced by any almost periodic process. This implies that 
for any ¢(£:), 71, the corresponding sample sequences have all their Bohr- 
Fourier coefficients vanishing except for the constant term. In particular, 
the sample sequences are not all uniformly almost periodic (or, for that 
matter, even Besicovitch almost periodic). Hence the transformation T itself 
cannot be almost periodic. That this is the case could have been deduced 
from the fact that neither JT nor any power of T is homotopic to the identity 
transformation (since the d; 40). But if T is almost periodic then some 
sequence, 7s converges uniformly to the identity, and since the homotopy 
classes of maps of K"— K" form a discrete set, this cannot happen unless 
eventually 7”) is homotopic to the identity transformation. 


vemark. Theorem 2.1 implies in particular that the transformations (2) 
leave no proper closed subset of K” invariant. It should be pointed out that 
for this weaker conclusion one can dispense with the Lipschitz condition on 
the g; (but not with the condition that the d; 40). In fact one can prove 
an analogue of Lemma 2.1 with strict ergodicity replaced by the condition 
that no proper closed sets are invariant. Here the condition on g(w.) becomes 
that equation (1) has no solution with R(,) continuous, rather than 
measurable. However, this is the case if g(¢,,- - -,&) is essential as a func- 
tion of For would have a certain degree in and one 
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verifies from (2) that TR(¢,,---,¢;) has the same degree in ¢;. Then 
+, would have degree 0 in and could not equal 
q(&,* * *, €j) which is essential in ¢;. Thus the smoothness condition on the 
qs in Theorem 2.1 is needed only to ensure that no measurable solution exist 


to (1). 
2.4. The sample sequence of a strictly ergodic process not only possess 
averages but these averages are taken on with a certain uniformity. More 


precisely 


uniformly in k as N->o. This is an immediate consequenc. of Theorem 
1.1. This notion occurs in [10] and [7] in the definition of a “ well-dis- 
tributed” sequence. A sequence {(n) on the unit circle is well-distributed 


if lim f(¢(n)) f(¢)dm(¢) uniformly in &, thus strengthening 
the for We might observe that the theorem ([7]) 
that if p(n) is a polynomial in nm with an irrational coefficient, then ¢?™#() is 
well distributed, is a consequence of Theorem 2.1. In fact, by an appropriate 
choice of integers 4; and a point w€ K", the sample sequence 7"f,(w) will 
have the form if the functions g; have the form 9j(€,,° 
+ The “well-distribution” of then follows from the 
fact that the unique stationary measure on K" is lebesgue measure. This 
takes care of the case that p(n) has its first coefficient irrational ; the general 


* case easily reduces to this one. 


3. Existence of ergodic averages. 


3.1. It is possible to show by examples that the conditions on the 
smoothness and the topological nature (i.e., the non-vanishing of degree) of 
the functions g; occurring in Theorem 2.1 cannot be essentially weakened. 
For example, if the Lipschitz condition is replaced by the requirement that 
the g; be of bounded variation, then the conclusion of Theorem 2.1 fails to 
hold, as a rather intricate construction shows. The necessity of the topological 
assumption is more evident since, e.g., the map: Tf, = = e?™ 
of the torus K? is clearly not strictly ergodic if p is rational. In the latter 
case, however, the ergodic averages, lim v3 T"f(w) still exist for all o, 
although they are no longer independent of a“ We will see in this section 
that the hypotheses of Theorem 2.1 are still relevant for the weaker con- 
clusion of existence everywhere of the ergodic averages. 
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THEOREM 3.1. Let (Q,7To,4o) be a strictly ergodic process, let Q 
=) X K and let T: AQ be defined by T (wo, £) = (Tow, g (0) €) where g 
is a continuous function from Q, to K. Then if the equation 


(1’) (wo) = R(T (wo) 


has a solution R(w.) which is measurable but not equal almost everywhere to 


a continuous functions, then lim N* } T"f(w) cannot exist for all continuou: 


n=0 
f and all w€Q. 


Proof. Let ¢(w) be a continuous function of Q) with yo(¢:R) AO, and 
consider the expressions T”"((w))£*). We have 


TEE = (wo) = R(T R(wo) or = (wo), 
whence 7" (€*R(wo)*) = so that 
(3) T" = ER (wo) (wo) (wo) ). 


Suppose now that all ergdoic everages exist for continuous functions on Q. 
Then the average of the left hand side of (3) exists, and so 


N-1 
* lim N* (wo) ) 
n=0 
N-1 
is defined. Since (Qo, To, wo) is ergodic we have 


almost everywhere. Since po(p'R) AO and (o,f) is a limit everywhere 
of continuous functions on Q, it follows that R(w.) agrees almost everywhere 
with a function R(w)) of Baire class 1 on Q, and hence on Qo. We shall show 
that this together with (1’) implies that R(w)) agrees almost everywhere with 
a continuous function, contrary to hypothesis. 

We say that a function f(w)) has an essential jump = 8 at a point wo, if, 
for any function f equal to f a.e., | —f(o”)|=s. Thus if 

w’ 


is not equivalent to a continuous function, then FR, and hence R, must have 
essential jumps = 6 for some 6>0. Now equation (1’) still holds a.e. for 
R and from it we see that the set of points at which # has an essential jump 
= 6 is invariant under 75. Moreover the set of points at which a function 
has an essential jump = 68 is closed. Now if (Q,7%,m) is strictly ergodic. 
every invariant closed subset of Q must contain the support of po. Since we 
may assume that the support of wo was all of Q, it follows that R(w,) has 
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N-1 

| 

| 
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essential jumps everywhere. But by Baire’s theorem, a function of the first 
class must have points of continuity ([4]). This proves the theorem. 

If the space Q» in the theorem is itself of the form (=, X K and T, 
has a form analogous to that of 7, and if g(o) =g/o1,€) is essential in €, 
then equation (1’) cannot have a continuous solution (cf. the remark in § 2.3). 
Thus, if in this case the ergodic averages exist everywhere, then 7 must be 
strictly ergodic. So, in the example referred to earlier, where the g; in 
Theorem 2.1 are of bounded variation, but do not satisfy a Lipschitz con- 
dition, and the process need not be strictly ergodic, neither will the ergodic 
averages all exist. 

Using Theorem 3.1 we can give an example of a map 7 of K? of the 
form (2), where 7’ is a C® transformation, but fails to satisfy the topological 
hypothesis, and such that the ergodic averages do not all exist. Define a 

1 
sequence of integers vz with v;=1 and =2*+u,+1. Set 
Writing n, = 2" we obtain 
9 


(3) | — [n.%|| > 


where [2] denotes the largest integer not exceeding .r. 


Define n and set 
h (0) = inne __ 1 ) 
k0 | k | 


and let g(e?™!?) = where is as yet undetermined. Note that 

is inessential. By (3), h(@) and hence g(¢) are C® functions. Now we have 

=H(6+«)—H/(6) where 


| k 


so that H(@) is in Z*(0,1) and, in particular, defines a measurable function. 


However H(@) cannot correspond to a continuous function since =) i | aliens 
and hence the series is not Cesiro summable at 60 ([11]). Therefore 
e*riM1(6) will not be a continuous function for some A. Taking R(e?*) 
= we have 


fi /R(L) = 9 


> } 
with R(€) measurable but not continuous. By Theorem 3.1, the trans- 
formation T of K? given by 


TE, = Tho = 9 


will not possess all its ergodic averages. 


"A 
V 
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3.2. The transformations given by (2) are in a “triangular” form, and 
this, we may note, implies certain restrictions in the homotopy classes of the 
transformations under consideration. Fixing our attention on the case r= 2, 
we note that the induced transformation on the 1-dimensiona! integral hom- 
ology group is given by a matrix 

1 0 
i 1) 


where d is the degree of g(,). Namely, the homology group is generated 
by the classes of the cycles p: = {£2—=1}, po= Under the trans- 
formation 

(4) 


= 


p; is transformed into p; + dps and pez into pz in the homology group. Now 
the homotopy classes of homeomorphisms of K? with itself are in one-one 
correspondence with the induced transformation of the first integral hom- 
ology group, and so may be represented by 2 X 2 unimodular matrices with 
integer coefficients. Thus the homotopy classes that are represented by the 
transformations in (4) are just those whose matrices have both eigenvalues 1, 
or equivalently, have trace 2. Now this restriction is quite natural if we are 
interested in strictly ergodic transformations, since if the associated matrix 
does not have trace 2, the map necessarily has a fixed point. This can be 
shown directly as a consequence of the fact that any two cycles on K? that 
are not homologous to multiples of the same cycle, must intersect. It is also 
a simple corollary of the Lefschetz fixed point theorem. However, a fixed point 
of the transformation is clearly not generic for a process unless the process 
degenerates to that one point, and so in any other case, the transformation 
cannot be strictly ergodic. 

The next theorem indicates that this restriction (with a slight modifica- 
tion) is also necessary for the existence of ergodic averages. We call a trans- 
formation of the torus linear if it is induced by a linear transformation of 
the plane. In order for this to happen, the associated matrix of the linear 
transformation on the plane must be a unimodular matrix with integer entries. 
It is easy to verify that this same matrix represents the homology class of the 
induced transformation of the torus. 


THEOREM 3.2. Let T be a transformation of the torus induced by the 

transformation 
(5) =ar+by+z2, 
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of the plane R?, where a, 6, c, d are integers and ad—be=1. Then 


N-1 
lim N-? S T"f(w) exists for every continuous function f on K? and w€ K?, 
n=0 


if and only if |a+d|S2. 
Proof. If a+d=2 then both eigenvalues of s=(“ 3) are 1 in 
which case, by a linear change of coordinates, (5) may be put into the form 


(6) y =de+yt+y’o. 


If d=0, or if d40 and 2’, is rational, it is easily checked that the sample 
sequences of the induced process are periodic or almost periodic. If d~0 
and 2’, is irrational then the ergodic averages exist by virtue of Theorem 2.1. 

If —2<a+d <2, then the eigenvalues of A are roots of unity and 
an appropriate power of A will have both eigenvalues 1. Hence a power of 7 
will fall into the category already treated, and the existence of ergodic 
averages for a power of T implies the existence of these averages for T itself. 

We may therefore turn to the case |a+d|>2. Here we wish to show 
that the ergodic averages need not exist. Since in this case the induced 
transformation of the torus has a fixed point, we may assume that the fixed 
point corresponds to the origin in the plane and that 7 in the plane is given by 


x’ =au-+ by 
y =cr+dy, ja+d|>2. 


-~2 
— 


We now consider 7 solely as a transformation of R?. We denote by Z* the 
lattice of points of R? with integer coordinates, and for a set A in R®, A+ 7? 
will denote the set of all points in R? congruent to a point in A modulo Z?. 
We let 7, and J, represent the lines through the origin containing the eigen- 
vectors of A; /, will correspond to the eigenvalue A, and J. to the eigenvalue 
A, Since |A+ A] > 2, A and A“ will be real and we take |A|>1. Also 
A is not an integer since A+ A™ is, and since A7— (a+djA+1=0, it 
follows that A is irrational. Hence the slopes of /, and /, are irrational and 
so the sets 7, + Z? and 1, + Z? are dense in R. 

We consider parallelograms whose sides are respectively parallel to 1, 
and /,; we call these oriented parallelograms. The “width” of the parallelo- 
gram will denote the length of the side parallel to /,, its “height,” the length 
of the adjacent side. TJ takes one oriented parallelogram into another one, 
multiplying its width by || and its height by |A|-*. Let Q, and Q2 be two 
oriented rhombuses with centers of the origin of R*, and with sides equal to « 
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and 2¢ respectively. «¢ will be determined later. To prove that not all ergodic 
averages exist for the induced transformation of the torus we will show that 
there exists a point » in R? with 
lim sup N-*{# of points 0 =n < N, such that € Q, + 
N> 


> lim inf N-*{# of points 0 = n < N, such that + 


Since Y, C Q2 this implies that for a doubly periodic continuous function f 
on #? that equals 1 on Q, and vanishes outside of Qs, lim NS T"f(w) does 
not exist. But this implies that for the induced transformation on K? and 
the associated function on K*, the ergodic average doesn’t exist at all points. 


To obtain the point » we prove the following two statements. 


(a) Given an oriented parallelogram P there exists an Ny such that 
for any NV > N, there is an oriented parallelogram P, C P with the property 
that for any w€ P,, the number of times Q.+ for OS n <N, is 
= fie N, being a constant depending only on A. 


(b) Given a noriented parallelogram P there exists an Ny such that for 
any N>WN, there is an oriented parallelogram P, C P with the property 
that for any »€ P, the number of times T"»€ Q,+Z? for OSn <N, is 
= (B2.+ B;loge*)-1N where 8, and depend only on A. 


The proof of (a) depends on the fact that the transformation 7 on the 
torus is ergodic with respect to lebesgue measure (as a straightforward Fourier 
series argument shows). Hence for almost all wo € R?, the frequency of entering 
Q.-+ Z* is the area of Q., which is proportional to «?, the constant of propor- 
tionality depending only on A. Hence for some wo in P, this will be true and 
the inequality in (a) holds for this point for every sufficiently large VN. But 
if it is true for w and a fixed N, it is also true for some neighborhood /; 
of w for that N. 

For the proof of (b), let 7,-+ v, be a translate of 1, by a vector vy in 7° 
which intersects P. For some positive integer mo, the parallelogram 7-""(), 
+v, will intersect P, thereby dividing P into 3 parallelograms, since 
tends to 1, as Let P, denote PN(T-™Q,+ Then 
T™(P,)C Qi:+2Z*. Consider an oriented parallelogram with center at % 
whose width is $ the width of 7-"-(), + v, and whose length is the length of 
T-"-Q,-+v multiplied by 8ye? where y is the sine of the angle between /; 
and /,. ye? is the area of Q,, and thus the area of the resulting paralelogram 
is 4. It follows by the Minkowski theorem on convex bodies, that this 
parallelogram contains a lattice point vo. Then it can be seen that T-"Q, +" 
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will intersect P,, dividing it into 3 parts, provided that | A |"~-™"°=> 16y~*e*. 


We may therefore take 
mM, = Mo + | (log | Bo’ + Bs’ log 


where 82’ and 8,’ depend only on A. We denote the intersection of T-™Q, + v4 
and P; by Then Q,+2?, T™(P.)CQ:4+ 2°. In a similar 
manner we can find a lattice point v, in a parallelogram related to T-™"Q, + 4 
in the same way that the foregoing parallelogram of area 4 was related to 
For this v. we will have cutting P. into 3 
portions, where m2, m,+8.’+8;loge’*. The intersection P; will satisfy 
(P;)C Q, + Z for i=0,1,2. Continuing in this way, we obtain parallel- 
ograms P, with T™(P,)C Q,+2Z? for 1=0,1,- --,n—1, and with the 
integers Mo, ™1,° *, separated by a distance not exceeding B.’ + B;’ loge. 
Now take B.—28.’, Bs =2B;’, and N =2m. The parallelogram P, is 
then chosen as the P, for my.< NS m,. For then, the number of times 
T*P, enter Q, + Z* exceeds n, and since N= < n(B.’ + log e*) 
< N/2 + n/2(B. + Bs loge), we have 


n > Bs loge?) 
Now suppose ¢ to have been chosen so small that 
Bie? < (B2 + Bs 


Let P be an arbitrary oriented parallelogram in the plane and form the 
parallelograms Pa, Pay = (Pa)», Pave = (Pav) c, etc., for a sequence of 
Take » as any point in the intersection of all the Pay.-ay. Since w belongs to 
infinitely many 


lim sup of On < N, such that T"w € + 
N-> © 
= (Bo + Balog e+). 
On the other hand since » belongs to infinitely many Pap... pa, 


lim inf # of OS n < N, with Tw € Q. + Z?} S pie’. 


N> © 


This » has the desired properties and this proves the theorem. 
Coronary. If satisfies where a is an integer, and 


N-1 
|A| > 1, then there exists a real number such that lim does 
not exist. = 


Proof. Let A -( “2 A has eigenvalues X and A-*. Let e, and e, 


2 


f 

| 
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, 
be two associated eigenvectors in a= es Now let 
1 


ye, be any vector in The ergodic 


averages of f(7"w) will exist for any continuous doubly periodic function f 


if they exist for f(u) = e?™#u'+lu") where u —(*,). Since A-"—> 0 it follows 


N-1 
that the ergodic averages exist for all f if lim N-* > e?™#A"rer'+12¢1") exists for 
n=0 


k, 1, and x. By the foregoing theorem this is not the case and hence with 
t = kere,’ + Ixe,” for and appropriate choice of k, 1, x, lim N-? will 
not exist. 

In fact, the conclusion of this corollary requires only the hypothesis 
that |A| > 1. This was pointed out to the author by Helson who has a very 
elegant proof of this result. 


4. Extensions of strictly ergodic processes. 


4.1. In §2 we studied processes that were defined inductively from 
lower dimensional processes in the following manner: To each point o, of a 
process (Q, 75,4) we associate the circle w) X K, and Q is taken as the union 
of these circles. A tranformation 7 is defined on Q by allowing T to permute 
the circles w) X K among themselves in accordance with Ty, and each circle 
is rotated by an amount depending on wo. In this section, we consider the 
more general situation where the transformation 7 on the individual circles 
is not necessarily a rotation, but any homeomorphism of the circle. 


Definition 4. A process (0,T,) extends a process (Q, (and 
(Q0, 7,40) is a subprocess of (Q,T,u)), if there is a continuous map ® 
taking © onto Q, with 67 —T,® and with po —®(,) in the sense that for 
any borel set A in Qo, wo(A) =p (@7(A)). 

We notice that any subprocess of (Q,7,) determines a subalgebra of 
C'(Q) by taking the set of functions f go. This subalgebra is isomorphic 
to C(Q,) and Ty on C(Q)) is induced by T on C(Q). Conversely one may 
show that any closed subalgebra of C(Q), invariant under 7, determines a 
subprocess. For example, the process (K*',7T,m") defined by (2) in §2 
is an extension of (K*,T),m*) for s <r defined by restricting (2) to the 
first s coordinates (¢,,:--,¢,;). The subprocess thus corresponds to the 
subalgebras of functions on K* depending only on (£,,- This sub- 
algebra is invariant under because each is a function of 


Definition 5. An extension (2,7,) of (Q, 7,0) is finite if there 
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is an integer h > 0 and a subset A of Q of measure 0, such that almost every 
point wo € 2) has at most h preimages in Q—A. 

If h=1, then (Q,7,p) is what has been called in [2] and L-extension 
of (Qo, T'o,40). This means that the continuous functions on the extension 2 
may be identified with measurable functions on 9. Conversely under these 
circumstances fh can be taken as 1. 

The transformation 7 of K? given by TE, 
leaves invariant a cycle on K?. Consequently 7 defines a finite extension of 
(K,Sa,m) where Sof = 

The following theorem can be regarded as a generalization of Theorem 1. 3. 


THEOREM 4.1. Let (Q,To,4o) be a_ strictly ergodic process, let 
Q2=Q) x K, and let T be any continuous transformaton of Q of the form 
T (wo, £) = (Two, 9 (wo, €)) where g: Then either T 1s a strictly 
ergodic transformation of Q or there exists an invariant measure p such that 
(0,T,) is a finite extension of (Q,T, 40). Moreover, in the latter case, 
every invariant measure p defines a finite extension. 


Remarks. If we define 6: by £) =o, then the condition 
on T is just that 7 should extend TJ), i.e., that P27 —7T,. Since @(m) will 
therefore be invariant under 7, if » is invariant under 7, and since po is the 
unique invariant measure for T,, it follows that under these conditions 
(Q,T,) necessarily extends (Q%, 7,40). We remark that Theorem 1.3 is 
obtained from this theorem by taking for (Q, To, uo) the trivial process for 


which 9, consists of a single point. 


Proof. Let M be the compact space of probability measures on K. An 
element of J/ is determined by a positive functional on C(K) taking the 
value 1 at 1. We shall need the following observation. If H is a dense 
subset of R(K), the set of real valued continuous functions on K, such that 
Hf is closed under additions, multiplication by rationals, and the lattice 
operations min(f,g), max(f,g), and f=1€ H, then any functional on H 
which is linear over the rationals, which takes non-negative values for non- 
negative functions in H, and which takes the value 1 for the function 1 
determines a unique measure in M. The reason is that any such functional 
can be extended to a linear, positive functional on C(K), as one may easily 
show. The significance of this is due to the fact that H may be taken to be 
an enumerable subset of C(K), and thus an enumerable number of conditions 
are imposed on a functional in order that it correspond to a measure in M. 

Now consider the subsets of Q9=Q, * K of the form AX K with A a 
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borel subset of Q,. These form a subfield B, of the borel field of Q, and so 
we may speak of conditional expectations with respect to By. By the fore- 
going remarks it will follow that there exists an M-valued function on 
Q4,4(m), such that these conditional expectations are obtained by 


(8) E(f|Bo) (wo) = (00, £) 


for any bounded borel measurable function f on Q. To construct p(w) we 
apply (8) to functions f(wo,¢) =g(£) where g€ H. The left hand side is 
defined almost everywhere and this defines a functional on H a.e. The fact 
that the functional may be defined on all of H a.e., is a consequence of the 
countability of H. Moreover from (8) if follows that at almost all ,, the 
functional obeys the enumerable set of conditions imposed in the preceding 
paragraph. Thus p(w.) is defined and it may be seen to be a measurable 
function from Q, to M with the borel field on MW induced by the weak topology 
on (i.e. the topology for which WV is compact: if pn(f) > for 
fe C(K)). It follows that (8) is valid for f depending only on ¢, and from 
this, (8) follows for f(wo,) of the form f;(o)f.(¢). It is clear then that 
(8) hold a.e. for any bounded borel measurable function on Q. 
From (8) and =F (f) we obtain 


9) Flot) du(oe) =f dao). 


We note also that (9) in turn uniquely determines the measurable function 
p(w.) up to sets of wo measure 0, as can be seen by taking f(wo, £) = f1(wo) f2(é). 

Let {1,2} denote, as in Theorem 1.3, the open arc on K from ¢; to ¢:, 
taken counterclockwise. Let us define an M-valued function p’(w.) by 


(wo) ( {£15 = (To) ({9 (wo, £1), 9 (wo; $2) }) 
where 7'(wo, £) = (Towo, (wo, £)). We then have 


Sit Jf (00) dy! (ws) 
Tol f (0, £)) du(wo) (£) }dpo (wo) 
% K 
=f (oo £) )dp.(wo) (£) }dpo (wo) f(T) du(o) 


But then p’(wo) p(w) a.e. so that we have: 


(10) ( {815 (Too) ({9 (wo; £1), 9 (wo, }) 
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almost everywhere. This says that T carries the associated measure p(w) 
on w) X K onto the associated measure p( Tow.) on Tow. X K. We shall refer 
to the (wo) as the relative measures. 

The condition that (Q,7,) be a finite extension of (Q, 70,40) may 
now be restated as the condition that for some h, the relative measures, 
(wo), have their support, for almost all wo, on finite subsets of K having no 
more than h points. If, on the one hand, (0,7,,) is a finite extension of 
(Q, To, 0), then the measure p’(wo) obtained by restricting p(w.) to the 
points of K corresponding to (w) X K) M (Q—A), must agree a.e. with 
p(wo) since p(A) =0. Hence p(w.) has its support a.e. on a set of Sh 
points. Conversely suppose that p(w.) has its support a.e. on Sh points. 
Define A as the union of the points (wo,¢) for which ¢ is not in the support 
of p(wo). We claim that A is a measurable set in Q. To see this, take I, 
a compact set in Q, with po(T.) >1—e, « >0, such that p(w.) is a con- 
tinuous function from T, to M. This is possible because of the measurability 
of w(wo). On T,, the support of p(w.) is a continuous function (with range, 
the set of unordered h-tuples of points of K). Hence ANT, X K is an open 
set and therefore measurable. But A—ANMI, XK has measure Se so 
that A is measurable. This type of argument may be used to verify a number 
of measurability assertions that we shall make. Now A must also have measure 
0. by applying (9) to the indicator function of A. But (w) X K)N (Q—A) 
has Sh points for every wo, and so ®""(w.) NM (Q—A) has Sh points so that 
(Q,7,p) is a finite extension of (Q%,7>,u). It should be pointed out that 
the set of points (wo,€) with ¢ in the support of »(.) may be much smaller 
than the support of ». Thus if each p(w.) is concentrated at one point 
£=¢(.), the support of » will be the closure of {(o,¢(o))} which may 
be all of Q. 

Now suppose that » is an ergodic invariant measure for 7, that does not 
define a finite extension of (Qp, To, wo) ; then a.e., w(wo) has infinite support. 
We claim that in this case, w(wo) is, a.e., a non-atomic measure, i.e. it has 
no discrete component. Otherwise define As(wo) = { (wo, £) : (wo) ({Z}) > 8} 


for > 0, and let As == LJ As(o). Again one can show that As is measurable 
Wo € 
by considering AsM (TX K) where T is compact and p(w.) is continuous 


when restricted to T. Now (10) shows that As is invariant under 7’; hence 
A; has either measure 0 or 1. This in turn implies that almost all As(o) 
have measure 0 or 1. The latter would imply that ~(.) has its support on 
=" points of K contrary to our assumption. Hence As(w.) has measure 0 
with probability 1, which implies that As(oo) is a.e. empty. Letting 8-0 
shows that (wo) is a.e. non-atomic. 
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Under the assumption that an ergodic, invariant p exists that does not 
define a finite extension, one can show that no other invariant measure y on 
(2,7) can define a finite extension. For suppose that y is an ergodic 
measure with v(#.) having its support on =A points for almost all op. 
The unordered h-tuple {£:(wo),° on which has its support 
is a measurable function of w). Note that h is independent of wo, since the 
set of w) having h points in the support of w(.) is invariant under Ty by 
(10). Now consider for each wo, the set of points 


A (wo) = {EE p(w) (wo), S< for some +=1,- +, h}, 


and take A= a (oo xX A(w)). Again A is a measurable subset of 2, and 


by (10), A is under 7. But by (9), eS She contrary to the 
assumption that p is ergodic, if 0<e«<h*. This shows that if an ergodic 
infinite extension exists, every extension is infinite. 

Now suppose that » and p’ are two invariant measures on (0, 7'), with 
p(w.) and p’(wo) non-atomic a.e. Let A=4(p+ yp’) so that A(wo) is again 
non-atomic a.e. Fix a point €€K and consider the map W from 
= X K-0’=Q, X K’ (K’ is the unit circle in the complex plane) 
given by 
(11) £) = (wo, , 


As in the proof of Theorem 1.3, we have that if £,, £., £3 € K, 
Ao) ( + A(wo)({E2, =A(wo)({E1, (mod 1). 


Hence we have 


(0, £)) = (Tow, 


(Two, (wo) Ao) (E {Sos §)} )) 


where g (wo) = e?™AMTow)({So9(@ok0o), the last equality being a consequence of 
(10). Hence, taking S(wo, £) = (Two, g(wo)£), we have 
(12) WT £) = SV (wo, £). 


Here it can be shown that the mappings W and S are measurable. In fact, 
on a compact set T relative to which A(wo) and A(T.) are continuous, the 
functions A(wo) ({£o, and A(T wo) 9 (wo, fo) }), entering in the defini- 
tion of W and S, are actually continuous. This is so because A(w)) is non- 
atomic. 

It is clear from (9) and (11) that © takes the measure A on Q into the 
product measure po X m on M’, in the sense that po X m(A) =A(¥4(A) ) 


| 
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equivalently, that A(foW) =po Xm(f). We write W(A) =po Xm. Let py 
and py’ denote respectively W(n) and W(y’), i.e. the images of » and p’ 
under the map We have then po X m=4$(py + yx’). But Tu implies 
v(Tn) so that by (12), S(¥(u)) or Similarly 
Spx’ =p’. Thus S is not an ergodic transformation for » Xm unless 

Assuming py; ~p,’ it follows that S is not ergodic. The Fourier series 
argument of Lemma 2.1 applies as well here (for g(o)) measurable rather 
than continuous), and we deduce that for some k 40, 


g* = /R (oo). 
For every interval A C K’, we may define 
A’(A) = { (wo, R(w.) € Aj. 


It is readily verified that po & m(A’(A)) =m(A). Now A’(A) is invariant 
under S, since if R(wo)*g*€ A then R( Tow) *Go* (wo) & € A which is just the 
condition for S(wo,£) € A’(A). Since m(A) is arbitrary, it follows that S 
has invariant sets of all measures between 0 and 1. From (12) it follows 
that if A’(A) is invariant under S, then A(A) = "(A’(A)) is invariant 
under 7’. Since W takes » into po X m it follows that there are invariant sets 
on Q of arbitrary A-measure. But we assumed that w and p’ were ergodic 
measures and A yp’). If a measurable set is invariant under it 
must have u-measure 0 or 1 and p’-measure 0 or 1. Hence it must have 
A-measure 0, 3, or 1. Thus we are led to a contradiction which means that 

From this we shall deduce that »— vy’. For this it suffices to show that 
=p’ (oo) But clearly the relative measures of and p’ are 
mapped into the relative measures of pw, and p,’ by W. Since p, —p,’, we 
have py (wo) —=py’(@o) a.e. To complete the proof it suffices to show that 
the relative measure is determined a. e. by its image py and similarly 
for w’. Here we are considering the relative measure p(wo) on wo X K, and 
by its image we mean the image under © on w) XK’. Now p(oo) and 
(wo) are both absolutely continuous with respect to A(wo) which is carried 
onto lebesgue measure on w) X K’. Jf an interval and a subinterval of w) XK K 
both map onto the some interval af w) X K’ under &%, the A(w,)-measure of 
the difference must be 0, by (11), and a foriori the p(w.)- and p’(w))-measures 
will be 0. It follows that the »(w))-measure of any interval of w) X K is the 
same as that of the preimage of the image of the interval under W, and hence 


| 
| 
| 
| 
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the same as the p,y(,.)-measure of its image under ¥. Consequently p(w) 
is determined by py(wo) and similarly for p’(w.). Therefore p—yp’. 

This completes the proof of the theorem. For we have shown that if 
there exists some infinite ergodic extension (0Q,7,y) of (Qo, To, o) then no 
ergodic extensions can be finite. In addition we have just shown that any 
other ergodic extension must be identical with the first. Thus unless 7’ is 
strictly ergodic, all ergodic extensions must be finite. 

The simplest example of this theorem for dimension > 1 is for O= K* 
and Tf, = e?™'*%,, Tf.=49(€:,f2). Our theorem asserts that there are two 
alternatives. Either T is strictly ergodic or all invariant measures on K° 
define finite extensions of (K,Sa,m) where Sof = e?™'*f. The latter alterna- 
tive implies in particular that for some h>0O there exists a measurable 
function A(é,) with its range in the space of unordered h-tuples of points on 
K, and with A(e?™#*Z) —g(f,A(¢)) = {9(€, 0’): € A(E)}. In particular 
for h =1, this would mean the existence of a solution to R(e?™#*¢) = g(¢, R(g)). 
The set A({) is here simply the support of the relative measure »(£) defined 
by the extension. Needless to say, in practice it is difficult to ascertain which 
of the two alternatives actually occurs. Theorem 2.1 gives the only cases for 


which we have resolved this question.’ 


5. A fixed point theorem and existence of an almost periodic sub- 
process. 


5.1. We restrict our attention once again to the two-dimensional torus 
Kk?, In the last section we saw that if the transformation T on K? has a certain 
form then some assertions could be made regarding the strict ergodicity of 7. 
We now inquire when a general transformation T of K? can, by an appropriate 
choice of coordinates, be put into the specified form. We are not able to 
fully answer this question ; however we can give conditions on T under which 
it will almost have the required form. To be precise, the condition on 7’ in 
the last section was that it define an extension of a 1-dimensional process 
(K,Sa,m), and that one of the two coordinate functions on K? can be taken 
for the coordinate function on K. What we shall obtain are sufficient con- 
ditions for the existence of a subprocess (K,Soa,m) for (K?,T,p), but the 
coordinate function for K may not be a possible coordinate function for K°. 
The reason is that a function from K? to K will be a coordinate function for 


1 Added in proof: There is a corollary to the proof of this theorem which is worth 
noting. In case Tf, = e"'*%,, Th =9(%&,%) does not define a finite extension of 
(K,Sq,m), then eq. (12) establishes a measure theoretic isomorphism between the pro- 
cess in question and the “standard ” process of eq. (2) (7 =2) in §2.1. 


STRICT ERGODICITY. 597 


K? only if its level sets are homeomorphic to a circle, and this need not happen 
for the mapping function from K? to K in our case. 

If a process (K*,T,) extend some (K,Sa,m) then by Definition 4, the 
associated map ¢: K?-—> K satisfies 67 — Sad. Looking at ¢ as a function 
in ((K*), #T is the function we have denoted 7’. Moreover, Sad = e?™'%, 
so that (K*,T,u) extends (K,Sa,m) only if there exists a solution to 
To = e?"'*g, i.e. a non-constant eigenfunction of T, in C(K?). Conversely 
if there exists such an eigenfunction ¢, then the algebra of functions of ¢ 
is invariant under 7 and so determines a subprocess which will be isomorphic 
to (K,So,m). The main result of this section is the following: 


THeEorEM 5.1. If T 1s a homeomorphism of K? leaving no proper closed 
subset of K? invariant and tf T ts not homotopic to the identity transformation 
then there exists a continuous eigenfunction ¢: Thb=e?"'*h where a is 
irrational. 

For the proof of this we shall require two preliminary results, the first of 
which is a fixed point theorem which is of some independent interest. 


Lemma 5.1. Let S: K*—K? be a homeomorphism of the form 

S86, = where G and H are continuous real 

valued functions on K*. Then if dA0 and sup | G(o’) —G(o”)| =7, 
€ K? 


then the transformation S must have a fixed point. 


Remark. The form of S as given here is the most general one for which 
S is homotopic to the linear map Sé, = £1, 8S, = 1%. We know from § 3 that 
up to a change of coordinates, this is the most general form which S can 
have if it is not to have a fixed point by virtue of the Lefshetz fixed point 
theorem. The present lemma gives conditions under which a fixed point must 
exist even if the homotopy class of S does not require the existence of a fixed 


point. 


Proof. Let p; denote the cycle {Az 1} and po, the cycle {{,=1}. By 
a cycle we shall always mean a continuous 1-1 image of A. To prove that 
there is a fixed point we shall show that for every « > 0 there are points w€ K° 
with | —£,(o)|<e and | —£.()| simultaneously. By 
compactness there will have to be a fixed point. 

Let T= {o: | Sl.(w) | <e} {w: | tH (S152) ] | <e}. On 
k*—T, the function ¢,4e?7/# (5) does not take on the value 1. Hence it is 
an inessential map and the function ¢,4 which is homotopic to it is aiso inessen- 
tial. Since d0 it follows that £, is also inessential on K?—T. Conse- 
quently on any cycle in T close to the boundary of I, ¢, is inessential. There 
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are now two cases to consider. Suppose first that there is some homologically 
non-trival (relative to K*) cycle in T arbitrarily close to the boundary of I. 
Since £, is inessential on this cycle, the cycle must be homologous to a 
multiple of p2. But a cycle homologous to lp, with |1| > 1, contains a cycle 
homologous to pz, so that there is in f a cycle homologous to p2. If, on the 
other hand, all cycles in T sufficiently close to the boundary of T are homol- 
ogous to 0 in K?, it is easily seen that T contains a cycle in every homology 
class and in particular one homologous to ps. Thus, in either case, [ contains 
a cycle y homologous to po. 

To show that there is a point » in with | —f£:(0)| 
| —£(w)| it suffices to show that | becomes <e 
on y. For this it suffices to show that for arbitrarily small §>0, G(o) 
oscillates on y by an amount > 1—6. Now suppose that on y, G({w) oscillates 
by =1—8 for some 8>0. Let [a,b] be the range of G(w) on K?, so that 
b—a=7, and let [a’,b’] C [a,b] be the range of G(w) on y, so that 
b’—a’=1—86. One of the intervals (a,a’) or (b’,b) has length = 3 + 8/2. 
Assume it is the first, the argument being entirely analogous if it is the 
second, so that a’ —a=3-+ 6/2. Let A be the set of points on A? for which 
G(w) lies in (a,a’). We shall show that A contains a cycle homologous to py. 
Since any cycle homologous to p; intersects any cycle homologous to pz it will 
follow that y must intersect A contrary to the supposition that on y, G(o) 
omits the values in (a,a’). This contradiction implies that G(w) does oscillate 
by >1—86 on y and the desired point w exists. 

We now lift the map S to a map S* of the plane. If we assume that 
the point (0,0) in R? corresponds to (1,1) in K?, then § will have the form 


== S*0, = 0; + G (6s, 62) 


(13) = S*6, = dé, + 6. + H(6,, 62). 


Let = denote the strip in R* given by 0=6,=1. A lifts to a set A* in the 
plane on which @(6,,0.) takes values in (a,a,). Hence A* 3% contains at 
least those points for which a+1=6,+ G(6,, 0.) <q, i.e., the points for 
which a+1=86,* <a,. To show that A contains a cycle homologous to pi, 
we will prove that for some 6, there is a curve joining (0,6.) to (1,02) in 3 
for which a+ 1< 6,;* <a,. Such a curve maps onto a cycle homologous to 


p: and lying in A. 

The function 6,* is well defined on R? and hence if a point (6,, 6.) belongs 
to the curve {6,*—(C}, then the point (8, + 1,6.) cannot also belong to this 
curve since 6,*(6,-+1,0.)=C+1 by (13). A less evident fact is that 
(6,-+ 4,0.) cannot belong to {6,*—C} if (0,,0.) does, for any (1. To 
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see this we note that if (6,,6.) and (6,-+ 4,6.) belong to {0,—C} for 
then since {6,* —C} is connected (since S* is a homeomorphism) it follows 
that for some (6,’, 62’) € {0,* = C}, we will also have (6,’ + 1, € {6,* =C}. 
From this we conclude that on a line segment: 0. = 6.’, 6,’ = 0, = 0,’ + 1, the 
function cannot takes values = 0,*(0,’,0.") +2 or (6, 62’) —1. 
For if at a point (6,, 62’) in this segment 6,*(61,02’) = (01, 02’) +2 then 
6,* (0, — 1, 6.’) = But every value of 6,* exceeding 
6,*(0,’ + 1, occurs at some (61, 02’), 0,” > 0,’ 1 and 6,” —(6,—1)=1 
since 6, = 6,/-+1=0,”. A similar argument gives the other inequality. 

Now in 3, @(@,,6.) attains its maximum and its minimum, so that 6,* 
must attain a maximum = 0b anda minimum =a+1in¥&. Also if S* isa 
homeomorphism, @,* cannot attain its maximum in & at an interior point of &. 
This implies that on the line 6,0, 6,* attains a minimum Sa-+1 and 
a maximum =—b—41. In particular, 6,* will vary between a-+1 and 
a+2+6/2<b—1 on 6,—0. There will therefore be an _ interval 
6. = on on which varies from a+1+8/4 to 
a+2+ 8/4 as 6. goes from 6, to 62°) and such that neither extreme value 
of 6,* occurs in the interior of this interval. 

Let A* be the relatively open set in & defined by the condition that 
a+1<6,* <a+3+8/2. A* C A* M3 sincea+3 +86/2 =a,. The inter- 
val on 6,0 from (0,02) to (0,0,°) lies in A*; let A,* be the connected 
component of A* containing this interval. We claim that the point (1, 6.°), 


which belongs to A* because 


6,*(1, 622) =14 6,* (0,6) =a +34 8/4 <a+34+8/2, 


actually belongs to Ay*. Suppose that this were not the case. Then one of 
the bounding lines 6,;* —=a-+1 or =a+3-+ 8/2 must separate (1, 0.) 
from (0,62) and (0,6.). But on the segment from (0,0.%) to (1, 62°), 
#,* cannot take a value less thana-+ 1 + 8/4, since 6,*(0, 6.) =a-+ 2 + 8/4. 
Hence the line 6,*—a-1 cannot separate the two points. On the other 
hand, the same reasoning shows that 6,* cannot exceed a+ 3-+ 8/4 on the 
line segment joining (0,6.) to (1,62.%). Hence the line 6,* =a-+3 + 6/2 
separating (1, from cannot pass below the level In order 
to separate (0,0.°) from (1,8.) it must however pass below the level 
»>=6,°), Hence on one of the lines 6,0, 1, it lies between the levels 
9. = 6, and @,°). This means that in the interval 6.0 < 6, < on 
#,* takes on either the value a+ 3+ 8/2 or a+2-+8/2. Both of these are 
impossible since in the interior of the interval in question 6,* <a+ 2+ 8/4. 
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This shows that (1, 62°) € Ao*, so that there exists a curve 8 joining (0, @,“)) 
to (1,6,)) lying wholly in A* C A*. It follows that 8 maps onto a cycle 
in K* homologous to p, and lying in A. This completes the proof of the 


lemma. 


Lema 5.2. Let T be a transformation of a compact metric space Q 
leaving no proper closed subset of Q invariant. If f€ C(Q), then a necessary 
and sufficient condition for f==Tg—g with g€ C(Q), is that the functions 
f+7f+-:--:-+Tf are uniformly bounded. 

This lemma appears in [3], p. 135 and, in slightly disguised form, in 
[2], p. 164. 


5.2. Proof of Theorem 5.1. From the fact that T has no fixed point 
we may infer that 7 has the form of S in Lemma 5.1, for some choice of 
coordinates (¢,,£.). In these coordinates 

TL, = 
TL, 


for some continuous real valued function H,, as may be seen by induction. 
Thus 7” also has the form of S, just with different values of d. Since no 
power of T has a fixed point if 7 leaves no proper closed subset of K? invariant, 
Lemma 5.1 implies that for every n, the function (@(T""w)+- - -+ G(o)) 
oscillates by <7 on Now for some o, limn1(G(T" +- - -+ G(o)) 
=a exists by the ergodic theorem applied to some stationary measure on K”. 
It follows that this limit exists and equals @ for all w in K°, and therefore for 
any invariant probability measure p, .(@) =a. Now take G’(w) = G(w) —«; 
then »(G’(T"*o) +: G’(o)) =0. It follows that in K*, G’(T"o) 
+:---+G’(w) cannot be everywhere strictly positive or strictly negative. 
Since this expression oscillates by <7 it follows that 


~ 


for all nm. We may now apply Lemma 5.2 since T leaves no proper closed 
subset of K? invariant. We conclude that there exists a continuous function 
Q(w) on K? with G@’(w) = Q(Tw) — Q(), whence G(w) = Q(Tw) — Q(o) + «. 
Hence TC, = and so that 
$(w) = f,e°7!9) is the eigenfunction sought. Here « must be irrational 
since if « =1/k, then the set of w for which ¢(w) = e?™”/k, y=0,1,- + -,k—1, 
would be a proper closed subset of K*. (Since ¢(w) is homotopic to &, it is 
an essential map and therefore ¢(w) takes on all values on the unit circle.) 
This concludes the proof of the theorem. 
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ON THE SUMMABILITY, BY SPHERICAL RIESZ MEANS, 
OF DERIVED MULTIPLE FOURIER SERIES.* 


By A. J. WHITE. 


1. Introduction. We suppose, throughout this paper that f(z) 
=f(2,,: is integrable in the Lebesgue sense in 0 = 2%; 2a (t=—1, 
- + +,k) and is periodic, with period 2z in each variable. The Fourier series 


of f(z) may be written 


where 
An, = (2x) f J f (x) dy, 
We write? 
A, (2) = Bea, 


n=ny2+ 
with the convention that A,(x) =0 if m cannot be expressed as the sum of k 
squares of integers. We form B,(a) from the series 


NK=- 


where the Dy,.--n, are arbitrary complex numbers, in a similar way. 


The spherical mean, f(t) of the function f(v) at the point 
= is defined by 


= wf + + tex) dog (k = 2) 
where o is the sphere é,?7-++- - --+ &°—1 and dog its (k — 1)-dimensional 


volume element. If then 
foo(t) = 3{f (2° + t) + f(a —Z)}. 
Riesz means S,°(f,2), of order § and type nm, of the series } A,(z) 


n=0 
are defined by 


* Received March 30, 1961. 
1The theory of spherical summability of multiple Fourier series was initiated by 
Bochner [2]. An introduction and an account of the main results is given in [7]. 
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R 
Sr°(f, v7) = 28/R? f (1— u?/R*)™uS, (f, w) du (8 > 0) 
= (8>0) 


The Laplacian A,%, defined by 


0° 0° 
1 . . 
A,**! = A,'(A,") (r == 1,2,° -) 
applied to the terms of the series >| An(a) yields 
n=0 
n=1 


Observing that 
= > (1 — n/R?)*ntA, (2), 
nSR? 
we may state known results concerning the summability, by Riesz means, of 
the series (1.1), in the following form. 


THeorEM A.” If there exists a polynomial P(t), in t?, of degree 2(q — 1) 


(= 0) such that 
(3k) 


as t-> + 0, then 
A,WSR°(f, 7) > s 


as whenever §>2q +4(k—1). 


THEOREM B.° Suppose that the trigonometric series >} B,(x) ts sum- 
n=0 
mable* (n,8) to s, for a non-negative integer 8, and that, for some « > 0, 


B,(0) = o0(n**). Then if g (>8+1) is a positive integer, and if 


g (2) k1(2q)! (—)72 n 1B, (2), 


* Theorems A and B are given by Shapiro ([9] Theorems 2, 3 and 5). We have 
altered his notation slightly. 
’ Shapiro has obtained a sharper version of this Theorem (see [10] Theorem 2). 
‘Tre. 3 (1—n/R*)5B,(@)> s as 
n 2 


= 
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there exists a polynomial P(t), in t?, of degree 2(q—1 ) such that 


st? (tk) 


9a(t) — P(t) ~ 
ast—>-+ 0. 


In section 3 of this paper we obtain, in Theorem 2, a generalization of 
Theorem A. We also obtain (Theorem 1) a converse result, which is a 
version of Theorem B applying specifically to multiple Fourier series. 

We also extend results recently obtained in [13] and show that the 
summability, by spherical Riesz means, of a derived multiple Fourier series 
depends on the behaviour of a certain one-dimensional Fourier series. Known 
results then enable us (at least when & is odd) to give a complete solution 
to the spherical summability problem for derived multiple Fourier series. 


2. Preliminaries. In this section we summarise known results, obtain 
some lemmas, and give definitions and notation which, together with that 
already given in section 1, will be required later. 


Bessel Functions. [11]. The Bessel function J,(¢), of the first kind 
and order » (= 0), is defined by 


Ty(t) = (1/2) (—)? 


We write, throughout this paper, 


Vu(t) = (t) (t>0) 
V,(0) =1. 
It is known that 
and that 
(2.2) (t) =—tV (2). 


Fractional Integrals. For t>0, £=1, € L(0,t) we write 


1 2) p-14,5-1 


= 
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and a similar notation is used when we replace ¢(u) by g(u) etc., with the 
convention that if ¢(u) —f,(u) and £=k we write for 


If, for q= 0, 


B(p, q + 36) gf2p+2a+h-2 


~ 2°T (p) 


as t—>-+ 0 we write® 


~ st™4 (Oz, p) 
as t> + 0. 


It is known that*® if p=1 
(2.3) | fo(t) | = O(1) (Ci, p) 
as t—>00. 

Summability by Riesz means. If &=0, and if 


(1—n/R?)®a,— s 


nSR? 
as Ro, then >a, is said to be summable (n,8) to s. 
Bochner’s Integral. Bochner’ has shown that if 6>4(4—1) then 


96- 8 1 


Since, by (2.3), | fe(t)| =O(1) (Ci, 1) as to it follows, using (2.1) 
that 


O(# 
+ | dt 


—0(R**), 


where «= 0, provided that §>a-+4k—4. Hence, in the case k —1, 


(2. 4) Sr°(f, x) = RS) fe(t) dé + o(R-), 


where B= 0, provided that 6 > £. 


5If €L this is equivalent to ¢(t)~ st*4(C,p) see [7]. 
* Bochner [2], page 185, (2.3) means 
u?) | f,(u) | du = (t-00). 
0 


* Bochner [2] page 176, Formula 10. 
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We require also the following lemmas. 


Lemma 1. [f r is zero or a positive integer, and tf S\n"ay ts summable 
n=1 


(n,8+r) (820) then Sa, ts summable (n,8). 
n=1 
This follows from the corresponding result, given by Bosanquet,* for 
summability (C), and the equivalence of the Riesz and Cesaro processes. 


Lemma 2.° If h is zero or a positive integer, and if 


Mu) 3 (—) 


v=q 


(n= 0) 


then 
1d 


where A is independent of x and t. 


Firstly for finite z and ¢ 


1d y(2v — 2q)° (2v—2q— 2h +2) 2y-2q-2 


where the differentiation under the summation sign is justified by the uniform 


convergence of the resulting series. Hence, for h—0,1,-- -, (2) 
is regular in the whole plane so that there exists a constant K, independent 
of z, such that 


(2.5) <K <1). 


Next, for u > 0, 
A(u) = 24u-*7V,,(u) —P(u) 


where 


Now by (2.1) and (2.2), for vw=1, h=0,1,- - -, 


1d 
u du 


(u) | < 24a-h-3 


(2.6) 


? 


where A is independent of wu. Further, for u=1, 


8 Bosanquet [4] page 277, Lemma 3. 
® Cf. also Shapiro [9], [10]. 
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|P(u)| 
p=0 2° + 1) 
(2.7) *) 
<= Au-?'-2 (h=0, 


Combining (2.6) and (2.7) we obtain, for w=1 and h=0,1,-- -, 


The required inequalities follow from (2.5) and (2.8). 


Lema 3. If p>4(k-+1) then for t>0 and all z, 


+ He) An (2) (EV). 


There exists a sequence {f"(x)} (r=1,2,---) of exponential poly- 
nomials such that 
(2.9) lim f | f(x) —fr(x)| da,- =0. 
If 
Ni(r) 
n=0 


then it follows from (2.9) that 


(2. 11) lim Dn, see An, ny 
uniformly in (,,° + -,”;,) and that, for each fixed positive ¢ and all « 
(2.12) lim = f2,9(¢). 


If N,(r) does not tend to infinity with r it follows that f(x) differs 
from an exponential polynomial only in a set of (k-dimensional) measure zero. 
In fact in this case the Fourier series of f(x) is finite and we may write 


No No 
n=0 n=0 


p.p.in 0S a,S 22 ((—1,:--,k). The result stated then follows on taking 


aA) 
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the spherical mean of order p of both sides of (2.13), since*® the spherical 
mean of order p of ei(m*+-+nerx) js, at the point 2, 


Now suppose that V,(r)—>0oo as roo. Taking the spherical mean of 
order p of both sides of (2.9) we have 


fo,p’ (t) = ny V 0), 
n= 
where c= (p+ 4k). Consequently, writing 
n= 
we have, for fixed r and N > N,(r) 


Ni(r) 


N 


n=N,(r)+1 
+ (t) 
Since as and since the 
number of points for which n,2-+- - is 


where £(> 0) is arbitrary, it follows from (2.1) that On, ny V pea (k-2) 1) 


is absolutely convergent for fixed t(>0) provided that p>3(k+1) + 2. 
Hence, from (2.1), (2.11), (2.12) and (2.14), for fixed t(>0) and all 2, 


Na(r) 
lim | Tx — fe,p(t)| | On, ny — | | (EV 0) 
Z n= 


+c | | | Voss (EV 2) | 


N,(r)+1 


+ | foo" (t) —fe(t)| 


=of{ > | (tV n) |} 


n=1 


* | Vora (¢V 0) |} 


+ 0(1) 
=0(1) 
as r—> oo, provided that p > 3(k +1) + 2£. 


1° See, for example, [6] page 994, Formula 10. 


e 
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Hence 
lim Ty = fz»(t), 
© 


for fixed positive ¢ and all x provided that p > 4$(k +1) + 2% Since 0) 
is arbitrary the result stated follows immediately. 

It is perhaps worth remarking that Lemma 3 may be put into a different 
form. We have, for p>4(k+1) andi >0, 


feo (t) —clim 


n=0 
N 
—clim { 2 Svn(f, 2) AL (tVn)] 


+ Svy(f, (EVN) } 
where Sr(f, x) = = An(Z). 


Hence, since = O(R*S) (where £(> 0) is arbitrary) it follows, 
using (2.1) that for fixed ¢(> 0) 


VN 
o (WS) | 
f RSn(f, 2) Vpsax(tR) aR, 


provided that p >4(k+1). In this form Lemma 3 has been obtained by 
Chandrasekharan 7! with the condition p > $(k + 1) replaced by p > $(k + 3). 


3. We suppose, throughout the rest of this paper, that q is a positive 
integer and that 


2°49 IT (q + $k) avt?” 
g(t) (4h) {foo(¢) ) (t>0), 
where the a, (v—=0,- - -,qg—1) are independent of ¢, and 2° is some given 


point. 
We first obtain 
11 [6] page 996, Lemma 2, the case = 0. Another form of this lemma is given in 


[5] p. 216 (and is reproduced in [7] p. 135) but the proof of this version appears to be 
invalid. 
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Turorem 1. If (—)?>.n%A,(2°) is summable (n,8+2q) (80) tos 
n=1 


then there exist constants a, such that 

(3.1) g(t) ~ (Ch, p), 

as whenever p> max[8—4(k—3),4(k+1)]. 
Moreover, for v=0,1,- - -,qg—1, 


(3. 2) ay = (—) sion 2 1 — n/R*) A, (2°). 

We may suppose, without loss of generality, that s=0O. For if, in 
particular, f(a) = and 2° = (0,- - -,0) then’ the derived 
series (1.1) consists of the single term se‘(*-+) which converges to s at 
x=2°. Moreover, if a,—=(—)’*4s then the a, satisfy (3.2). Since the 
spherical mean at x= 2° of js (Lh) Vy (x2) (t) 


it follows that, in this case, 
q-1 (—) 


y=0 


~~ 


ast—>-+0. Hence, if f(z) is replaced by — (—) 


replaced by zero. 
Writing 
T(0,u)—= 
4? 


we note that since A,(x°) = O(n**-"*5) where £(>0) is arbitrary 
(3.3) T (m, wu) = O (m=0,1,-- -). 
Writing also, for simplicity, 7 =6-+ 2q, »=p+4(k—?2), 
y(t) = (t) 
+ 1) p(t) = 
it follows from Lemma 3, (2.6) and (3.3) that, for fixed ¢ > 0, 


(3.4) 


12 For a similar argument see Bosanquet [4] Lemma 8. 


| 
| 
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= ZntA. (2®)y(tVn) 


R 
= t*4lim lim n%A,(2°) ey (¢V n) 


y>+0 Row n=1 


VR 
—#¢lim lim (— uT'(0,u)(~ (ut) du 
1 


y>+0 R> 


(3.5) (tv B)} 


— #24lim lim { > (— ym T(m, u)(~ (ut) ]u=-VR 


y>+0 R>0 m=0 


+ ful (ut) du} 


1 
=== (—_.) = 
= (—)"t?4lim + (ut) du 


y>+0 


1d 
= (—)"#4 lim + uT(h +1, ujew— 


y>+0 p=0 
We now choose h to be the greatest integer less than +. Then since** 
T(h+1,u) it follows by that T(h + 1, u) 
=o(u?"2) and hence, using (2.6), that for »y—=1,2,---,4-+2 and fixed 
0), 


uP © du 


= f, dy 

as y>+0, if p>é—4(k—3). Hence from (3.5), if 
p > max[3—4(k—8), $(k+1)], 


(3.6) = (—)*#2¢lim UT (ut) du. 


y>+0 


We now write, as in Lemma 2, 


co y2"-24 
—2 22%) IT (v + p+ 1) 


= 3 


18 Throughout what follows ¢ , ¢, * denote constants not necessarily the same 
at each occurrence. 
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and 
q-1 
where 


IT 
Substituting the relation 
y(u) =P(u) +A(u) 


in (3.6) we obtain 


q-1 
p(t) = Ddc,t?” + (—) lim f uT (h +1, y (ut) du 
p=0 1 u du 


y>+0 


(3.8) 


say, where for | 0, q— 1, 
*b, lim T(h +1, du 
1 


(3.9) 
f T(h 41, u) u2¥-24-2h-3 du 


since (T(h+1,u) =O(u*"**)) this last integral is absolutely convergent. 
We first show that 

(3. 10) I(t) =O(1), 

as t>-+ 0. 


If x is an integer, then s=h-+ 1 and so by Lemma 2, 
= 1 d 
I(t) = ‘tim uT (7, — (ut) du 
1 u du 
(—)7 w) d t)d 


on f du) + O (f [ (ut) -#-24-7-3 +. (ut)-?7-4] du) 
t-1 
—0(1) 


as t—> +0 provided that p > 8—4(k—3). 
If r is not an integer, we write 


T(h+1,u) =e (x, 0) dv 


y=0 
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where ¢ is independent of wu. We then substitute this expression into the 
integral defining J(¢) and invert the order of integration. Estimating the 
inner integral by standard methods then leads to (3.10).** 

Collecting results we have, from (3.8), 


q-1 
= Zest” + 
i.e., using (3.4), that 


q-1 
f2°,p(t) IT (y+ 1) + O(#2) 
where 
dy = IT y+ 1) (v=1,-- -,g—1) 
= Co + (w+ 1). 
Now 
9 -2p-k+2 t 
so that | 
($k) 


ag 


+ 


= —T(u+ 1) 1) 
—0(¢2) 

as t> +0. 


We have thus shown that if (—)?.n%A,(x°) is summable (n,8-+ 2q) 
n=1 


to s then there exist constants a, such that (3.1) is satified for 
p > max[8— }(k—8), $(k+1)]. 
It remains to show that the a,, which are clearly unique, satisfy (3.2). 
It follows from Lemma 1 that (—)* > nt1A,(2°) is summable 
(n,8-+2q—1), i.e. that 


n=R? 


** We omit the details, which are rather tedious. We shall only use Theorem 1 in 
the case when 7 is an integer. The method outlined here is a combination of those in [5] 
page 215, Verblunsky [11], and [3]. 
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say. From what we have already proved (with q replaced by g—1) there 
exist constants a,’ such that 
ast? 


(4k) 221 (v + $k) 
~ 


} 


as t—>-+0. However we have just shown that 


(q + 4k) 
(3) 


22 


dyt2” 

IE (v + $k) 
IT (q + $k) 

24-2 (g—1) IT(q—1 + $k) 

(q + $k) 

—~ + 2?4-2(q—1) IT(q—1 + $k) 


(f.0(t) —T(4k) 3 


= g(t) 9 


2q-2 + $k) 
as t—+>+0. Hence a,y—a,’ (v=0,1,- --,qg—2) and i.e. 


lim > (2°). 


n=R? 


Repetition of this argument shows that all the a, satisfy (3.2). 
We next obtain a theorem which is converse to Theorem 1. 


THEOREM 2. If constants a, exist such that 


(3. 11) g(t) ~ pp) (p20) 


as t>+0 then ts summable (n,d+ 2q) to s whenever 
n=1 
8>p+3(k—1). 
Moreover the constants a, are given by (3.2). 


We observe first that we may suppose, without loss of generality that 
s=a,=0 For in the case 


f(z) =P(z) = da,{ (ti? — tx)" 


(78 Rar 5 t= ,k) 
where it follows from a known theorem’ that (—)’ is 
n=1 


summable (n,8-+ 2v) (8 >4(k—1) to a, at (v=0,: - -,@q). 


‘© [7] page 124, Theorem 4. 62. 


t 
| 
| 
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Moreover in this case** g(t) =st?4 (0<t<7) so that if =P(z) 
then for any p= 0 (3.11) is satisfied, the conclusion of Theorem 2 is satisfied 
and the a, satisfy (3.2) so that by considering f(x)—P(a) in place of 
f(x) we may suppose s=0 =a, -,q—1). 

We may thus suppose that 


(3. 12) (t) = 0 
as t—> +0. 
Next" 
q 
(3. 13) 2°) = R2 (Ff, 2°) 
y=0 
where the cy (v=0,---+,q) are independent of R. Hence it is sufficient 


to show that 


From a known result 
0 
provided that > p+4(k—1). Consequently, from (3.12) and (2.1), 


R98 p5+?a+v (f, 2°) ( fk+2p+2q-1 dt) 
0 


J na 


= 0(1) 

as R-»oo provided that §>p+4(k—1). 

Finally it follows from Theorem 1 that, since (—)¢S)n2A,(2°) is sum- 

n=1 

mable (n,8-+ 2q) to s, the a, are given by (3.2). 

4, Throughout this section we suppose that h(t) is an even periodic 
function with period which coincides with t*1g(t) in 0 < < and that 

h(t) ~4b, + cos nt. 
n=1 
The main result of this section may then be formulated as follows. 


** It is easy to verify that in this case 


t) =T(ak) 
fa(t) = & aR) 


17 [9] page 227 
*® [10] page 214. 
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TueorEM 3. If 8>4(k—1) (k>1) and q 1s a positive integer then 
(2°) is summable (n,8-+2q) to s if and only tf there exist 
constants dy (v=0,- -,g—1) such that (i) g(t) as t> +0 
and (ii) if k is odd Sin**4b, is summable 
4 b—1) or (ii)! is even 

—> + k—1) 2q + $k + 4) 
as 

The a,, when they exist are given by (3.2). 


We require two further lemmas. 
Lemma 4. If 5>4(k—1) then 
as Ro. The same is true if g(t) ts replaced by t74, 
Since, by (2.3), as 


0(1) (Ci, 1) + O(#*) 
= 0 (t?4-*) (C;,1) 


it follows, using (2.1) that 
| at | (g(t) | 


as R— oo, since > 4(k—1). 
Finally, by (2.1) 
— 

if }>4(k—1). 


1° Equivalently (—)*% 3 n%A,(«@°) is summable by Riesz means of some order to $. 
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LemMA 5. In order that be summable (n,8-+ 2q) 
(83 >4(k—1)) to s is necessary and sufficient that there exist a, such 
that (1) g(t) ~st*4(C;,) as t>-+0 and (ii) 


+3) 
Rees g dt (§ +1) 


as R00. 

Necessity. Suppose that (—)?>)n%A,(2°) is summable (n,8-+ 2q) to s. 
It follows from Theorem 1 that constants a, exist such that g(t) ~ st?4(C;,) 
asto>+o0. If 


P(x) = —}(dq= 5) 
(2° 2° + - -, 


then, as in the working for Theorem 2, when f(2) —P(2) the ay satisfy 
(3.2) and is summable (n,8+ 2q) (8 >4(k—1)) tos. 
Moreover, since g(t) =st74 (O0<t<7) it follows that (i) is satisfied. 
Condition (ii) is satisfied since, using Lemma 4, 


0 

0 

70 

s V5, (th) dt + 0(1) 

0 
4k)” 


Hence by subtracting P(x) from f(x) we may suppose that s—=a,—0 
(v=0,- - -,qg—1), i.e. it is sufficient to prove that 


(4.1) th) at — 
as Roo. 
We observe that, by Bochner’s formula, (4.1) is equivalent to 
(4.2) (f, x°) = 0(1) 
as Ro, 


7° [11] page 994, Formula 10. 


then 
> + () 
k 
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Since f.0(t) = O(t*4) (C;,) as it follows, as in the working for 
Theorem 2 that, for some 7», 


(4.3) R48 p7**4(f, 2°) =O(1) (r= 7) 
as Roo. Further, by (3.13), 

v=0 


Since, by hypothesis, the left hand side of (4.4) is O(1) as R—>oo when 
t= 54 it follows by successive applications of (4.3), (4.4) and the consis- 
tency of Riesz means that (4.2) is satisfied. 

Sufficiency. We may suppose, as before that a,—s—0 (v=0,1,---, 
q—1) and that (4.1), or equivalently (4.2), holds. The result then follows, 
by consistency from (4.4) with 7 replaced by 8. 

Proof of Theorem 3.7+ Firstly, we may suppose that s=0. For in the 
case when 

—P(o) — 


we have, by arguments similar to those used in Theorem 2, that 
is summable (n,8-+2q) (8 >4(k—1) to s, and if we 
put a,=0 (v=0,1,---,qg—1) that the a, satisfy (3.2). Moreover in 


this case, for 0 << ¢t<2,g(t) =st™%. Hence, if & is odd, (—)9*#¢-» > 


is the (2g -+4&—1)-th derived Fourier cosine series at of and 
is therefore summable (C,8+ 2¢-+4k—4) to (2q¢g+h—1)!s whenever 
8 >4(k—1) ((14), p. 60). On the other hand if & is even a straightforward 
calculation of the Fourier cosine coefficients b, of s | t?4**1| shows that 
neath, —> (—)Ba(2q + k—1) !28/m, (C,84+2q+ 4k +4) (8>4(k—1)). 
Thus when f(z)—P(r), is summable (n,8-+ 2q) to s 
for any 6 >4(k—1) and (i), (ii) or (ii)’ are satisfied. Hence if we con- 
sider f(x) — P(x) in place of f(x) we may suppose that s 0. 

Necessity. The case k odd. We suppose that k is odd, § > 4(k—1) 
and (—)?3'n%A,(2°) is summable (n,§+ 2q) to 0. It follows from Lemma 
5 that constants a, (v=0,1,---,q—1) (satisfying (3.2)) exist such that 
g(t) =O(t2)(C,) as t-> +0, i.e. that (i) is satisfied, and that 


Roark (t) dt = 0(1) 


* The proof of this theorem is similar to that of Theorem 1 of [13], we give brief 
details for completeness. 


a 
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as R—o, and hence by Lemma 4 that 


(4.5) f dt = 0(1) 
0 


as R-> 
Now by (2.4), with B= 2q -+4—1 and 6 replaced by 8+ 2¢ + $k—4, 
f by h we have 


Tv 
0) f h(t) dt + 0(1) 
0 


(4.6) 
o(1), 
by (4.5). 


But, since & is odd, 


q+4(k-1) 
(1 n?/R? ) 6+2q+hk-} (n?/R?) > n? 
v-0 


so that 
q+h(k-1) 
(4.7) S(1—n2/R2) — 0) 
n=R 


=o0(1) 
as Ro, by (4.6) and consistency; equivalently }n74**-1b, is summable 
(C,8+ 2q¢ + 4k—4) to zero. 
The case k even. This is proved in a similar way (cf. (13)) and we 
omit the details. 
Sufficiency. The case k odd. Suppose that there exist constants a, 


(v=0,---,q¢g—1) such that g(t) and that is 
summable (C,8-+ 2q 4k—4) to zero or equivalently that 


> (1 n?/R? ) 5+2q+B(k-1) 2q+k-1H ( 1 ) 
n<R 


It follows from (4.7) and consistency that either 


(4. 8) R2atk-1g (h, 0) 0(1) 
as or that for some sequence 7; << << tp, 
(4.9) (h, 0) A0(1) (r= 


as R->«. However, since g(t) = 0(t?4)(C;,) as t—> +-0 it follows from (4. 6) 
(with § replaced by +), and arguments similar to those in the proof of Theorem 
2, that for some ry (h,0) = O(1) 70) ie. that (4.9) 


= 
is- 

le 
t 

e 
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is false. Hence (4.8) holds and so by (4.6) and Lemmas 4 and 5 }n44,,(2°) 
is summable (n,6-++ 2q) to zero. 


The case k& even is proved in a similar way. 


5. Supplementary remarks. Condition (ii) of Theorem 3 may be re- 
phrased as “the (2¢ + &—1)-th derived Fourier series of h(t) is summable 
(C,8+ 2g + 44—4) to (2g at t=0.” Combining this observa- 
tion with known results in the case k =1 ([3] Theorem 2, [13] Lemma 2) 
we obtain the following result: 


THeEorREM 4. If §>4(k—1), where k is an odd positive integer, then 
(—)¢Sn2A,(x°) is summable (n,8+ 2q) to s if and only if there exist 
constants ay (v=0,: -,qg—1) such that (i) ts integrable in 
the Cesaro-Lebesgue sense in (0,7) and (ii) its Fourier cosine series 1s sum- 
mable (C,8—4k+ 4) tos at t=0. 


If they exist, the constants a, are given by (3.2). 
For details of a similar calculation we refer to [13] Theorem 2. 


It seems likely that the case & even of Theorem 3 can be reduced in a 
similar way, but the required one variable results are not known. On the 
other hand the imposition of additional conditions on f(a) enables us to 
obtain various simplifications of Theorem 3 (case k even). We finally give a 
simple and useful result of this kind. Here we write w(t) for the even 
periodic function which coincides with st?4**-? in 0<t<-z. We also write 


(5.1) X(t) =h(t) —o(#) 
and suppose 

(5. 2) X(t) + yn cos nt 
(5.3) w(t) ~ $8 + } 8, cos nt. 


We then have: 


THeEoREM 5. If §>4(k—1) where k ts an even integer and if there 
exist constants ay (v==0,: -,q—1) such that t-°9*X(t) € L(0,7) then in 
order that (—)2>n%A,(z°) be summable (n,8-+ 2q) to s it is necessary and 
sufficient (i) that the (2qg-+hk—1)-th derived allied series of the even 
function X(t) be summable (C,8+ 2q¢-+4k—4) at t=0 or equivalently 
(ii) that the allied series of the odd periodic function equal to t-*4-***X(t) 
im 0<t<z, be summable (C,8s—4k+4) at t=0. 


oc 
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If they exist the constants a, are given by (3.2). 
Proof. It is easily verified that ¢-°4*x(t) € L(0,7) implies that 


t 
f X (uw) du = 0(t?4*) 
0 


as +0, and hence, from (5.1), that g(t) ~st*4(C,,1) as t>-+0. 
Consequently the initial assumption of Theorem 5 ensures that condition (i) 
of Theorem 3 is satisfied. Moreover a straightforward calculation shows that 
for 8>4(k—1) n?a**§, (—)#*9(2q + + 2g + 344+ $4) 
so that from (5.1), (5.2), (5.3) 


—» (—)#+9(2¢ + hk —1) !28/r(C,8+ 2¢ + 4+ 4) 
if and only if 
(5.4) O(1) (C,8-+2q + 


as n—>oo i.e. condition (ii)’ of Theorem 3 is satisfied if and only if (5.4) 


holds. 


Sufficiency of (i). Suppose that }n7#**1y, is summable (C,8-+ 2q 


n=1 
+ 44—4) then n4*y, =0(1) (C,8+2¢+4h+4) ie. (5.4) holds so that 
condition (ii)’ of Theorem 3 is satisfied. Since condition (i) of that theorem 
is also satisfied it follows that (—)¢}}n4%A,(2°) is summable (n,8-+ 2q¢) to s. 


Necessity of (i). Suppose that (—)4 > n¢A,(2°) is summable (n, 8 + 2q) 
to s then condition (ii)’ of Theorem 3 is satisfied and hence (5.4) holds. 
Since ¢-*4-*X(t) € L(0, x) it follows [3] that the (2g + &—1)-th derived allied 
series at t= 0 of X(¢) is summable (C) i.e. } n74**ty, is summable (C). This 
together with (5.4) implies that > n*4**-1y, is summable (C, 8, + 2¢-+4k—#). 

The equivalence of conditions (i) and (ii) follows, in virtue of the 
integrability of t-°4-*X(t), from a result due to Bosanquet [3]. 
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THE COHOMOLOGY OF DIFFERENTIABLE TRANSFORMATION 
GROUPS.* * 


By Ricwarp 8, Panars and THomAs EK, STEWART. 


1. Introduction. If a Lie group G@ acts differentiably on a manifold 
In then various spaces of tensor field on 9 become in a natural way modules 
for the Lie algebra § of G and the cohomology of 9 with these coefficient 
modules in certain cases carries interesting information about the action of 
G. In this paper we will discuss this situation, at first in a somewhat more 
abstract setup, and develop a method for computing these cohomology groups 
in certain cases. In particular we shall show that if @G is compact and semi- 
simple then even though these modules are infinite dimensional the conclu- 
sions of the First and Second Whitehead Lemmas [10], [11] are valid; namely 
the first and second cohomology groups are trivial. As one consequence we 
will show that differentiable actions of compact, semi-simple Lie groups admit 
only trivial infinitesimal deformations ($11) a fact whose global analogue 
will be found in [7]. Our second and motivating application of these general 
cohomology results is to a question initiated by one of the authors in [9]. 
Namely if a Lie group G acts differentiably on the base space of a differen- 
tiable fiber bundle B over VW can G be made to act differentiably on B so 
as to be equivariant with respect to the fiber projection and so that each 
operation of G on B is a bundle map. We show here that the answer is yes 
if G is compact and simply connected and if the structural group of B is a 
solvable Lie group, and moreover that the way of “lifting” the action of G 


to B is essentially unique. 


2. Cohomology and invariant cohomology of Lie algebras. In the 
following L is a finite dimensional Lie algebra over a field F of characteristic 
zero. For the defintion of an Z-module and a complete discussion of the 
cohomology of L with coefficients in an Z-module we refer the reader to [2] 
or [3] (we shall use the notation of the latter). The notion of the invariant 
cohomology of L with coefficients in an [-module is implicit in a number of 
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papers, however the lack of an exposition with the relevant facts that we 
shall need makes the following discussion desirable. 

Recall that if VW is an L-module then the space C?(L,.V/) of p-cochains 
of L with coefficients in M is defined to be M if p—0 and to be the space of 
alternating multilinear maps of LZ? into M if p is a positive integer. Then 
C*(L,M) the cochain complex of L with coefficients in VW is the graded 
vector space € C?(L,M) with the differential d of degree + 1 defined by 


p=0 


dm(X) = X-m for m€ M and 
(1) 
+2 (— 1 ) Xj], Xi, X,, X;, ) 


for c€ C?(L,M), p>0. We recall from [3] that d? 0 so there are defined 
graded vector spaces Z*(L,M) and B*(L,M) of cocycles and coboundaries 
and their quotient H*(L,M) the cohomology space of L with coefficients 
in MW. We recall also that C*(L,M) becomes a graded L-module with 
M=C°(L,M) as a submodule if we define 


(2) q ‘ 
i=1 


for c€ C?(L, M), p> 0, and that 

(3) d(Xc) =X (dc) 

for Xe L, c€ C*(L,M). It follows from (1) and (2) that for c€ Ce(L, 1) 


A 


We define a cochain c€ C*(L,M) to be invariant if Xc=0 for all X€ L. 
It follows from (3) that the graded subspace C;*(L,M) = @ C,°(L, M) of 


C*(Z,M) consisting of invariant cochains is stable under d and so gives 
vise to the graded vector spaces Z;*(Z,M) and B,*(L, M) of invariant cocycles 
and invariant coboundaries (N.B. B;*(Z,M) =d0,*(L,M) not the possibly 
larger C;*(L,M)N B*(L,M)) and their quotient H;*(Z,M) which we call 
the invariant cohomology of L with coefficients in M. By the natural homo- 
morphism of H;*(L,M) into H*(L,M) we mean the homomorphism induced 
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by the inclusion of C;*(L,M) in C*(Z,M). We note from (4) that 
= d|C;*(L,M) is given on C?(L,M) by 


(5) —2 (—1)*4e([ Xi, Xj], X1,° X 5° X per) 


An L-module M is called trivial if Xm =0 for all YE L, me M (i.e. if 
C,°(L,M) =C°(L,M) =M). Any vector space V over F' can be considered 
a trivial Z-module, in which case we write dr for the differential on C*(L, V). 


Clearly from (1) 


i<j 


In particular we regard F as a trivial L-module and write C*(L, F) 
—(*(L) and H*(L,F) =H*(L). Since C1(L) = L*, the dual space of L, 
and since by (6) for ce Ct(L) it follows that 
2 (L) =[L, L]°, the annihilator of [Z, LZ]. Since B*(L) is clearly zero (for 
df(X) =0 for f € F) it follows that H1(1) = Z*(L) = [L, L]° = (L/[L, L])*. 
Thus H1(L) is trivial if and only if Z is its own commutator subalgebra. 
Note also that H* is an additive functor, i.e. H*(L,M @ N) = H*(L, M) 
@ H*(L,N), so it follows that if H?(ZL) —0 then H?(L,V) =0 for any 
trivial finite dimensional L-module V. 

If M is any L-module then for each X € L there is an endomorphism ‘tx of 
C*(L, M) homogeneous of degree —1 defined by ixm =0 for m€ M=C%(L, M) 
and (txc) (X1,° °,Xp1) =c(X, +, for c€ Co(L,M), p>0. For 
later reference we recall from [3] that the module operations of Z on C*(L, M) 
is related to d and the operations tx by 


(7) Xc = + ixde X€ L,c€ C*(L,M). 


3. An extension of the Whitehead lemmas. We will show in § 8 that 
if & is the Lie algebra of a compact Lie group then for a certain class of 
S-modules the natural homomorphism of H;*(9,M) into H*(G,M) is 
an isomorphism onto. If % is semi-simple it follows from the theorems we 
are above to prove that for such §-modules H'(%,M) and H?(%,M) are 
trivial, a fact that would also be a consequence of the Whitehead Lemmas 
({10] and [11]) if W were finite dimensional. It is in this sense that this 
theorem extends the Whitehead lemmas. 


THEOREM. I[f L is a finite dimensional Lie algebra such that H'(L) =0 
then Hp (L,M) =0 for all L-modules M. If in addition H?(L) =0 then 
M) =0 for all L-modules M. 
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Proof. C)°(L,M) = M|Xm=0 for all X € L} so if me C/°(L, M) 
then dm(X) =Xm= 0 for all L hence B(L,M) =0 and H7'(L, 
=Z)(L,M). By (5) of §2 if ce Cf(L,M) then de(X, Y) VY ]) 
so is zero on [Z,L]}. But H*(L) =0 is 
equivalent to L=[Z,L] and hence implies MW) =Z/(L, M) =0. 

Now let c€ Z;°(L,M) and let V be the finite dimensional subspace of MV 
spanned by {c(X,¥Y)|X,Y¢€L}. We consider V as a trivial L-module. 
Assuming H?(L) =0 it follows that H*(L,V) =0. Now ce C?(L,V) and 
comparing (5) and (6) of §2 we see that drc—=— djc=0 so c€ Z°(L,V) 
= B?(L,V) and hence C= d,76 for some C1(L,V) CC*(L,M). It will 
suffice to prove that 6€ C/(L, 1), for then comparing (5) and (6) of §2 
again d;(—06) =—d,6 =d76=c so c€ B;?(L,M) and we will have shown 
=BP(L,M). Now if X,X,,X.€ LZ then from (2) of $2 
(Xc) (X,,X.) —=X(c(Xi, —e([X, Xi], X2) —e( Xi, [X, 


=X (6([X1,X2])) —O([[X, 41], X2]) [X, X2]]) 
and by the Jacobi identity and the linearity of @ it follows that 
0= [Xi,X2])) — LX, [X1, X2]] ) 


so, referring to (2) of §2 again, (X@)([411,X.]) 0. Thus X@ vanishes 
on [L, LZ] so @ is invariant. q.e. d. 


4. Topological G-Modules. In this and succeeding sections G will 
denote a compact Lie group, G, its identity component, and 9 its Lie algebra 
of left invariant vector fields. By a topological G-module we shall mean a 
complete, metrizable, locally convex, real topological vector space (a Frechét 
space) J together with a fixed homomorphism T of G into the group of 
automorphisms of J/ such that for each m€ WM the map g>T(g)m of G into 
M is continuous. We denote the space of continuous linear functionals on MV 
by M/* and we write <m,l> for 1(m) if (l,m) € M* & M (which of the many 
possible topologies to put on J/* is irrelevant for our purposes and we shall 
always regard M* as untopologized). In general we will drop explicit 
reference to J and simply write gm instead of T(g)m. Note that since J is 
a Frechét space and for each m€ M the orbit {gm|g€ G} is compact, and 
hence bounded, it follows from the principle of uniform boundedness that 


4.1. THEoREM. Given a neighborhood V of zero in the topological 
G-module M there is a neighborhood U of zero in M such that gué€ V for all 
(g.u)€G@XU. Equivalently if {m,} ts a sequence in M converging to m 
then {gm,} converges to gm uniformly for g€ G. 
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Noting that g,m,— gm = g,(m,— m) + (g-m— gm) it follows from 4.1 
that if g,—>g then g;m,— gm. 


4, 2. Corottary. If M is a topological G module then (g,m)— gm ts 
a continuous map of GX M into M. 


Now let W be a topological G-module and M* the linear space of functions 
from a compact Hausdorff space X into M. We give M* the topology of uni- 
form convergence, i.e. a typical neighborhood of zero is {f€ M*|f(X) CU} 
where U is some neighborhood of zero in M. By a step function from XY to M 
we mean an element f € M/* whose range is a finite subset {m,,-- -,m,} of 
M such that each f-*(m;) is Borel measurable. Since a continuous function 
from X to M is uniformly continuous it follows easily that the space C(X, M) 
of continuous maps of X to M is included in the closure of the space S(X, M) 
of step functions. Given a Radon measure » on X and f¢€ S(X, MM) we define 
ffdp — 2 ef? (m) )m. Then f— ffd» is a continuous linear map from 


S(X,M) into M and so (because M is complete) extends to a continuous linear 
map of the closure of S(X,M) into WM. Henceforth we shall only be con- 
cerned with { fd» when f is continuous and we shall need the following obvious 
facts. 

(a) uniformly on Y¥> ffadu— 

(b) ffdp is bilinear in f and p 

(c) If =1 then closed convex hull of f(X). 

(d) If 7 is a continuous linear map of M into a complete topological 

vector space then f (7f)du=T ffdp. 


The space C(G) of continuous real valued functions on G is a Banach 
space in the supremum norm and becomes a topological G-module under the 
operations given by If f€ C(G@) we define f*m, for m 
an element of a topological G-module /, to be the integral of the continuous 
map g—>f(g)gm of G into M with respect to normalized Haar measure on G. 
If 1€ M* then by (d) above <f*m,l>—= ff(g)<gm,l>d-g. From this and 
the invariance of Haar measure it follows that <(gf)* m,ly = <qg(f*m),1y 
and since 1/* separates points of WM it follows that (gf)* m—=g(f*m). 


4.3. THroreM. If M is a topological G-module then 
(1) (f,m)—-f*m ws a continuous bilinear map of C(G)X& M into M 


(2) If f€C(G) ws positive and ff(g)dg=1 then f* m€ closed convex 
hull of {gm|f(g) 49}. 


| 
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(3) If me M then f>f*m is an equivariant continuous linear map 
of C(G) into M and m is in the closure of its range. 


Proof. Statement (1) is immediate from (a) and (b) above and state- 
(2) follows from (c). The equivariance of f—>f*m (i.e. the fact that 
(gf)* m =g(f*m) was shown above so it remains only to show that if W 
is a closed convex neighborhood of m in M then there is an f€ C(G) such 
that f*meW. Let U be a neighborhood of the identity in G such that 
gm€ W for g€ U and let f be a continuous positive function of integral one 
on G with support in U. Then by (2) fe me W. q.e. d. 


5. Almost invariant vectors. An element m, of a topological G-module 
M is called almost invariant if {gmo|g€ G} spans a finite dimensional sub- 
space of WM. We denote the set of almost invariant vectors in M by ,. 
Clearly M, is a subspace of M/ invariant under G, and in fact M, is just the 
linear span of the finite limensional G-invariant subspaces of MV. An almost 
invariant vector in C(@) is called an almost invariant function on G. From 
the bilinearity of (f,m)—f#*m and the equivariance of f—>f#*m it follows 
that if f, is an almost invariant function and m any element of a topological 
G-module M then f,*m€ My. Now the Peter-Weyl theorem [1, p. 203] says 
that the almost invariant functions are dense in C(@). Since we known that 
m is adherent to {f*m|f¢ C(G@)} and that f>f*m is continuous we have 


the following essentially known result. 


5.1. THrorem. In any topological G-module M the subspace M, of 


almost invariant vectors is dense. 


5.2. Tueorem. If M is any topological G-module then the subspace 
M, of almost invariant vectors has a §-module stducture defined by 
Xm = lim 1/t((ExptX)m—m) for X€ Y, me My. 
t>0 


Proof. If V is any finite dimensional invariant subspace of WM, (g,v)— gv 
is an analytic map of G X V— V (this follows for example from [1, Prop 1, 
p. 128]) so that the indicated limit exists for m€ V. That this gives a §- 
module on V follows from |[1, Theorem 2, p. 113]. Since M, is the linear 
span of its finite dimensional invariant subspaces the theorem follows. 
q.e. d. 


6. Differentiable G-modules. 


6.1. Lemma. Let M be a topological G-module and {m,} a sequence 
in M converging tom. Given X € & andlé€ M* suppose there exists a sequence 
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{ux} im M converging to w such that d/dt< (Exp tX) m,, ly = (Exp tX),;, l>. 
Then d/dt< (Exp tX)m, ly = (Exp tX)o, ld. 


Proof. By 4.1 gm, converges to gm uniformly in g, so since 1 is linear 
and continuous and hence uniformly continuous, <gm,,l> converges to <gm, l> 
uniformly in g and a fortiorn <(Exp tX)m,,1> converges to (Exp tX)m, I> 
uniformly in ¢. Similarly <(Exp ¢tX)o,,/> converges to <(Exp tX)o,/> uni- 
formly in ¢t. By a theorem of elementary calculus this validates differentiation 
“under the limit sign” and gives the desired result. q.e. d. 


6.2. THrorem. Let M be a topological G-module X € &, and suppose 
there is a function ¥: M—>M such that 


d/dt<(Exp tX)m, ly = <(Exp tX)Xm, ly 


for all m€ M and all 1 belonging to a separating family SC M*. Then X is 
a continuous linear map and for me M, Xm =lim1/t((ExpTX)m—~m). 
t>0 


Proof. The linearity of X is immediate from the linearity of differen- 
tiation and the fact that S is separating. The preceding lemma gives 
immediately that XY has a closed graph and, since M is a Frechét space Xx 
is continuous. If m€ M, then by 5.2 1/t((Exp tX )m—m) converges strongly 
and a fortiori, weakly to a limit say m’. By definition of Y <m’,l) = <Xm, 1) 
for 1€ S and again since § is separating m’ = Xm. q.e. d. 


6.3. THroremM. Let M be a topological G-module, 1 € &, and suppose 
that the linear map 


m— lim 1/t( (Exp tX )m—m) 
t>0 


of M, into itself (see 5.2) ts continuous. Then this map extends uniquely 
lo a continuous linear map ¥ of M into itself and d/dt<(Exp tY)m,1> 
= <(ExptX)Xm, ly for all me M, le M*. 


Proof. The existence and uniqueness of X follows from the completeness 
of M and the denseness of M, in M. Given] € M*, m€ M,, and ¢, a real number 
l,: m—> <(Expt.X)m,l> is an element of /* and 


< (Exp tX) mo, = <(Exp(t + to) mo, ld 
dt | t=to db | 4-0 


(Exp tX) mo, lo> = <X mo, lo> = (Exp ) Xmo, ld. 
at t=0 

Thus <(Exp tA’) m, is differentiable for m€ and M* and its derivative 
is <(ExptX)Xm,ly. If m is an arbitrary element of M choose a sequence 
{m,} in converging to m. Then Ym,—>Xm and for any M* 
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d/dt< (Exp tX )m,, ly = (Exp t¥)Xm,,ly so by 6.1 d/dt¢(ExptX)m, ly 
= ¢(ExptX)Xm, ly. q.e.d. 
6.4. THroreM. If M is a topological G-module then the following are 


equivalent conditions. 
(1) For each X€ & there is a function X¥:M—M such that for all ] 


in a separating subset of M* <(Exp tX)m,l> =< (Exp tX)Xm, for ail 
M. 

(2) For each X € & the map X°: m—> lim 1/t( (Exp tX)m—m) of My 
into itself (see 5.2) 1s continuous. ~ 

(3) For each XE and me M t— (Exp tY)m has a weak derivative, 
X'm, at 

(4) For each X€ GY there is a map X*: M—M such that 


¢(Bxp (X)m,1> — <(ExptX)X*m, 1p. 


Moreover if any one, and hence all, of these conditions are satisfied then 
the maps XY, X’, and X* are all equal and are the unique continuous linear 
map of J into intself which extends X°. 


Proof. It is clear that (4) implies (1) and that if XY and X* exist that 
they are equal. From 6.2 it follows that (1) implies (2) and that if 
exists, it is the unique continuous linear extension of X°. From 6.3 it 
follows that (2) implies (3) and that X* is the unique continuous linear 
extension of 1°. It remains to show that (3) implies (4) and that V’” = 1%. 
Assuming (3) holds let 7€ 1/* and ¢, a real number and define /, € M* by 
m—> <(Expt)X)m,l>. Then for any M 


< (Exp tX) m, ly = ¢(Exp((t-+ t))X)m, ly 
t=to t=0 


| 
| < (Exp tX) m, = <X’m, = (Exp t.X)V’m, ly. 
( i=0 
This shows both that (4) holds and that V*m=.X’m. 
(|. e. d. 


6.5. Definition. A topological G-module is called differentiable if it 
satisfies any one and hence all of the properties (1)-(4) of 6.4. 


Remark. It is easily shown by example that the maps t— (Exp 4t.V)m 
need not be strongly differentiable for all m in a differentiable G-module. 


\ 


n 
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6.6. THrorem. If M is a differentiable G-module then M has a §- 


module structure which is characterized by the identity 
d 
<Xm,l>= =| <(ExptX)m, lb 
dt | s=-0 


forX€Y, me M,1€ M*. Moreover each of the module operations of & on 


M is continuous. 
Proof. Immediate from 5.1, 5.2, and 6.4. 


6.%. Definition. If M is a differentiable G-module then the $-module 
structure for M described in 6.6 is called the derived §-module. 

Henceforth differentiable G-modules will be regarded without explicit 
mention as ¥-modules, the derived 9-module always being understood. 


6.8. THroreM. If M ts a differentiable G-module then: 
(1) << (Exp tX)m,l>y =< (ExptX)Xm, Y, me M, le M* 


(2) (ud(g)X)m, XE g€G,meM 

(3) If me M then Xm=0 for all X€ Y if and only if gn=m for 
all gE Go. 

Proof. (1) follows easily from 6.4 and the definition of the derived 
module structure. To prove (2) we note that if € 1/* and we write 


then 
<(ad(g)X)m, <Exp t(ad(g)X)m, l> 
t=0 
= d | 
<g(Exp tX)g-*m, ly 
dt | t=0 
<(ExptX)(g%m),l 
dt | t=0 
== <Xg-1m, ly = <gXg"'m, l>. 
If gm =m for all g€ G then (Exp for all € and all real ¢ 
hence for each VY € §¢— (Expt) has strong and therefore weak derivative 
zero at ¢==0 and by Definition Ym —0. Conversely if for a given 
& then by (1) <EXp tX, =0 so <(ExptX)m, ly =< (Exp 0X) m, 


=<m,l>, (ExptY)m==m. Since G@ is compact every element of Gp is of 
the form Exp.Y and (3) follows. q.e. d. 
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7. The differentiable structure of C*(9,M). In this section we will 
show that if M is a differentiable G-module then C*(%,M) is in a natural 
way a differentiable G-module also. Moreover we shall show that the derived 
%-module structure for C*(%,1/) coincides with the other “natural” §- 
module structure that it has gua cochain complex of the -module M (§1. 
equation (2)). Finally we shall show that the differential d is continuous 
on C*(%,M) and commutes with each operation of G. 

Since C*(9,M) = @C?(8,M) and C?($,M) =0 for p> dim §, to 
define a differentiable G-module structure on C*(%, 1) it suffices to define 
one on each 0C?(9,M), p> 0. Fora typical neighborhood of zero in C?(%, 1) 
we take {c€ - -,Xp) €U if - -,X,€ B} where B is a 
compact subset of 9 and U a neighborhood of zero in M. Thus ca—c means 
that for each compact subset B of M ca converges uniformly to c on B?. The 
metrizeability, completeness, and local convexity of C?(%,M) follow directly 
from the corresponding properties of /. The operations of G on C?(%, 1) 
are defined by (gc) Xp) -,ad(g*)X,p)). It is 
obvious that each such operation is an automorphism of C?(%,M/) and that 
(91g2)€ =g:(goc). That g— ge is a continuous map of @ into C?(§, 1/) 
for any c€ C?(%,M) is a straightforward argument which we leave to the 
reader. Given X,,---,X, in and a 1€ -, Xp), is an 
element of C?(9, M))* and the collection of such continuous linear functionals 
on C?(%, 1) is clearly separating. Now put 


g(t) =c(ad( Exp —tX)X,,- - -,ad(Exp—tX)X,). 
From the well-known fact that 


lim (ad( Exp tX) —ad(Expt,.X)Y) =ad(Expt.X)[X, Y] 
t> tp 40 
it follows from the multilinearity of c that g(¢) is strongly differentiable at f) 


and that 
(to) =— c(ad( Exp — -,ad(Exp— t,.X)[X, Xi], 
iA 
+,ad(Exp —t,.X)X>). 
(We have used the fact that a multilinear map of a finite dimensional 


space into a topological vector space is automatically continuous.) Putting 
A(t) = (Exp tX)g(t) we have 


—x(t)) - ((Exp tX)g(to) —g (to) ) 


+ (Exp tX) (g(t) —9 (to)) 


0 
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By the joint continuity of GX C?(%,M)—>C?(&,M) the second term con- 
verges to (Expt,X)g’(t.) as ¢ converges to to while by (1) of 6.8 the first 
term converges weakly to (Expt,)¥g(0). Thus if /€ M* then 

=| <((ExptX)c)(Xi,° =a} 

dt | t=to dt | t=to 


= (Exp toX) (Xe) ) Xp), 


where Ye € C?(%, 1) is defined in (2) of §1. It follows that condition (1) 
of 6.2 is satisfied by the topological G-module C?(,1/) and hence that 
is a differentiable G-module, the derived 9-module structure more- 
over being that given in (2) of $1. If {cy} is a sequence in C?(G, J) 
converging to c then it is clear from (1) of §1 that for any V,.- - -.\p. 
dea(X1,° *;Xp.1) converges in M to de(X,,- -.Xp), from which it is clear 
that the map d: C?(%,M)—C»(9,M) has a closed ‘graph and so, the 
domain and range being Frechét spaces, d is continuous. A similar argument 
shows that for each Y€ & the linear map iy: 0?(9,M)—>C?(G,M) is 
continuous. 

From (2) of (6.8) it follows that Vgc=g(ad(g")X) for all g€ G, 
Ye and ce C*(¥,M). Using this and equation (1) of $1 a straight- 
forward computation gives g(dc) =d(qgc). 


8. The main theorem. 


8.1. Lema. If M is a differentiable G-module then each operation of 
on C*(, is chain homotopic to the identity, t.e. for each g € Gy there 
isa linear map (g) of C*(9,M) into itself such ge—c=dd(g)c +A(g) de 
for all c€ C*(¥,M). Moreover, the map g—>2A(g) can be cosen so that 
g—>X(g)c is continuous for each c€ C*($,M). 


Proof. Since G, is compact and connected we can find, for each g € Go, 


71 
X€ § such that g—expX. We define A(g) -¢= (Exp tX)ixce-dt. Since 
0 
C*(%,M) is a differentiable G-module from 6.8, (1) for any 1/€ (C*(9, M))* 


<gc—c,l>= <(Exp tX)c, l>dt 
0 


( (Exp tX)Xc 
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and recalling that Xc = dixc + ixdc (§1, (7)) 


<ge—c,l> = <f (Exp tX)dixe +A(g)de, ly. 
0 
Since d commutes with Exp ¢X and integration we obtain 


<gce— ce, l> = <dr(g)e + A(g) de, ld 


This equation holds for all 1€ (C*(9,M))* and hence A(g) is a chain 
homotopy. 

Now suppose that U is a small enough neighborhood of the identity so 
that the exponential map has a continuous inverse, f. Define then 


f (Exp f(g) dt 


do(g) has the required properties and g—A,(g)c is continuous on U. Now 
choose *,gn in G so that giV cover and choose A(gi) satisfying 
the lemma. It is easily seen that if we define A;(g) for g=guwegU by 


=A(Gi) + Ao(U) + A(Gi) dro (U) + AA(Go) Ao (U) 


then gc —c=(ddA;(g) + A(g)w)c and g—>A,(qg)c is continuous on giU for each 
c€ C*(Y,M). If we put a partition of unity refining 
the covering {g\U}, for g¢ giU then clearly g>¢i(g)Ai(g)e is continuous 


on Gp», so if we put A(g)c = > gi(g)Ai(g)c then g—>A(q)c is continuous on 
i=1 
U, and satisfies the requirements of the lemma. 


8.2. Lemma. Let po be the normalized Haar measure on Gy. If M isa 
differentiable G-module, define A: C*(&, M)—> C*(G, M) by Ac= fgcduo(g). 
Then there is a linear map X of C*(G,M) into itself such that Ac—c 
= + dAdc for all c€ C*(G,M). 


Proof. Define Ac= f (g)¢dpo(g) is chosen as in Lemma 8. 1. 


8.3. Lemma. Let M be a differentiable G-module and let A: C*(§%, M) 
— C*(%,M) be the linear operator defined in Lemma 8.2. Then A has 
the following properties: 


(1) Jt is a projection of C*(9,M) on C;*(9.M). 
(2) It commutes with d. 


(3) If 2€Z*(Y,M) then Az—ze B*(G,M) i.e. Az is cohomologous 
to 2. 
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Proof. Property (1) follows directly from the invariance of Haar measure 
and statement (3) of 6.8 (applied to C*(%,M) instead of WM). Statement 
(2) follows from the fact that d commutes with the operations of G on 


C*(%,M) and with the integral. Finally (3) is an immediate consequence 
of Lemma 8. 2. q.e. d. 


8.4. THe Main Turorem. If M is a derived &-module then the 
natural homomorphism i*: H;*(%,M) H*(G&,M) induced by the inclusion 
map of C7*(G, M) in C*(G,M) ts an isomorphism onto. In other words every 
cohomology class of H*(4,M) contains an invariant cocycle, and two invariant 
cocycles which differ by a coboundary differ by the coboundary of an invariant 


cochain. 
Proof. Immediate from 8.3. 


8.5. Corotuary. Jf & is semi-simple then H'(%,M) = H?(9,M) =0 
for all derived $-modules M. 


Proof. Immediate from 8.4 and the theorem of § 3. 


9. The differentiability of tensor modules. Let 9 be a differentiable 
(f-space, i.e. 9M is a differentiable (—C*) manifold and there is given a 
differentiable map (g,p)—>gp of GX ™M into In (the action of G on M) 
such that ep==p and (gg')p=g/(g'p). For each X€ & there is a differen- 
tiable vector field Y* on 9 defined by X*, is the tangent to t— (Exp — tX) p 
at t=0. The map 1—X* is a homomorphism of % into the Lie algebra 
of differentiable vector fields on 9% which is called the infinitesimal generator 
of the action of G on I. Let J be the space of all differentiable tensor 
fields on 9M of any fixed (mixed) rank and symmetry type with the usual 
“C*-topology ” (i.e. convergence means uniform convergence of each com- 
ponent and of each partial derivative of any order of a component on any 
compact subset of a coordinate neighborhood). It is well-known that J is 
a complete locally convex space and in fact a Montel space. Moreover if In 
is second countable, as we henceforth assume, then J is metrizable. Each 
diffeomorphism ¢ of 9 induces an automorphism of J in a well-known 
way and we will write this automorphism as ¢ also. Moreover if {¢,} is a 
sequence of diffeomorphism of 9m converging to a diffeomorphism ¢ in the 
(*-topology then it is clear that for any J. By a theorem 
of Montgomery [4] if we write g* for the operation p—gp of an element 
of G on In then g—g* is continuous from @ into the group of diffeomor- 
phisms of 9 with the C*-topology. It follows from these last two facts that 
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J is a topological G-module if we define gf =g*T. We shall call such a 
topological G-module a module of tensor fields (associated with the differen- 
tiable G-space 9) and shall as usual write gT instead of g*T. It follows 
easily from Theorem II of [5] that a module of tensor fields J is always 
differentiable and that for Y € § and 7'€ J XT is just the usual Lie derivative 
of T with respect to the vector field ¥*. We can now forget about the topology 
on J and even the action of Gon J. All that is important for the applica- 
tions we shall make is summed up in 


THEOREM. Let 9n be a differentiable G-space and X > X* the infini- 
tesimal generator of the action of G on MM. Let JF be the space of all 
differentiable tensor fields on In of a fixed rank and symmetry type, and for 
X€Y and Te TJ let XT be the Lie derivative of T with respect to \*. 
Then this makes J into a derived -module so that (8.4 and 8.5) H*(%, 7) 
= H,*(&,F) and if & is semi-simple H*(9, 7) =H?(%,F) =0. 


10. H*(G,R) = H*(%) as a special case. Consider G as a differ- 
entiable G-space under the map (g,p)— pg. Then for each € J the 
associated vector field Y* on G@ is just X itself. Let J be the tensor module 
of all differentiable real valued functions on G. If » is a p-form on G then o 
defines an element c of C?(9,F) by c(Xi,- = w2((X1)2,° (Xp) 
Conversely given c€ C?(9,7) define a p-form wo on G as follows: given 2 € G 
and tangent vectors Y,,- - -,Y, at x let w,(Y;,: Vp) = c(Xi,- +, Xp)(2) 
where Y; is the element of 9 satisfying (Xi), =Y;. It is readily verified 
that » is differentiable and that these two maps are mutually inverse linear 
isomorphisms between C*(%, 7) and the space 2)(G) of differentiable forms 
on G. Moreover it is a well-known fact (a proof can be found in [15]) that 
under this correspondence the differential on C*(%, 7) corresponds to the 
exterior derivative on Q*(G). Thus by deRham’s Theorem H*(G, ?) 
= H*(%,J) and so by the theorem of §9 H*(G,R) =H;*($,7F). For 
c€ C?($,7) we have 


(X1,° +,Xp) (x) =e(ad(g*)X1,° -,ad(g*) Xp) (2g) 


and if X € § then by left invariance (ad(g-*)X)2g—=dRg(X~) (where Rg is 
right translation by g). It follows that c€ C;*(%,i) if and only if the 
corresponding form w is right invariant. This gives the well-known identifica- 
tion of H*(G,R) with the cohomology of right invariant forms (form of 
Maurer-Cartan). On the other hand we see that each c€ C?(%) corresponds 
to a unique c’€ such that -,Xp) +, Xp) (e); 
namely - -,Xp)(g) =c(ad(g)X.,: - -,ad(g)X,). This sets up a 
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linear isomorphism between C*(%) and C;*(%, 7) and referring to equations 
(5) and (6) of § 1 we see that dy corresponds to — d; under this isomorphism. 
It follows that H,*(%,7) =H*(%) and this gives the well-known result 
H*(G,R) =H*(G%). In [2], where the cohomology theory of Lie algebras 


| was first made explicit, there is a detailed account of theorems of this general 


nature. 


11. Infinitesimal deformation of differentiable G-spaces. Let %n be 


| a differentiable G-space, ®: IM M the action of G on In and Y—> X* 


the infinitesimal geenrator of @. Suppose that for each ¢¢ [0,1] J there 
is given an action of G on In such that and svch that p,¢) 


| is a differentiable map of GX IN XI into Such a family 


, will be called a deformation of & and we write Y — X;,* for the infinitesimal 
generator of ®,. It is easily seen that (1, p,t) —> (X¢*)» is a differentiable 
map of & X nN XI into the tangent bundle of 9% and it follows that for 
each Y€ Y there is a differentiable vector field DX on % such that 


(DX)p— = (X;*),. Clearly D is a linear map of 9 into the Lie algebra 
t=0 


Y of differentiable vector fields on % and since [X,Y ]¢*—=[4X:*, Y:*] it 
follows that D({[X,Y]) =[DX,Y*|+[X*, DY]. We call D the infinites- 
imal deformation of ® associated with #, and in general a linear map D”’: 
$—Y satisfying the above identity is called an infinitesimal deformation 
of 6. Now if ®, is a deformation of 9 (i.e. (p,t)—> ®(p) is a differentiable 
map of Im I—M and for each is a diffeomorphism of the 
vector field Z defined by Z,—tangent to t+ ;(p) at {0 is called the 
infinitesimal deformation of 9m associated with ¢;. From ¢; we can construct 
a deformation ©, of ® by (9, p) =¢:(®(9,¢r'(p)). A deformation of ® 
that can be defined in this way is called trivial. It is easily seen that the asso- 
ciated infinitesimal deformation D of ® is given by DX = [Z, X*] =ad(Z)X* 
where Z as above is the infinitesimal deformation of 9m associated with q. 
In general if Z’€ UD’: X >ad(Z’)X* is an infinitesimal deformation of ® 
and we call such infinitesimal deformations of ® trivial (at least if n is 
compact Z’ is the infinitesimal deformation of 9 associated with some 
deformation ¢; of 9 so in this case D’ is the infinitesimal deformation of ® 
associated with a trivial deformation of ©). Now consider VY as a tensor 
module associated to the G-space 9 and recall that the Lie derivative of 
Ye Y with respect to Z€ V is just [Z, Y] (see, for example [5] Lemma c). 
It follows that Y is a derived § module under the operations YY = [X*, Y]. 
Now an element of C1(%,Y) is just a linear map c: § > V and c€ Z’(Y,V) 


| 
| 5 
| 


638 RICHARD S. PALAIS AND THOMAS E. STEWART. 


if and only if 0=dc(X,Y) =Xc(Y) —Yc(X) —c([X, Y]) i.e. if and 
only if c([X, Y]) =[X*,c(Y)]+[¢(X), Y*]. Thus Z’(9,V) is just the 
space of infinitesimal deformations of ®. On the other hand cé€ D’(%,Y) 
if and only if for some Z€ Uc(X) =dZ(X) =X(Z) =[—Z,X*] ie. if 
and only if c is a trivial infinitesimal deformation of @. Thus H1(9,V) =0 
means every infinitesimal deformation of © is trivial. Since UV is a tensor 
module it follows from the Theorem of § 9 that 


11.1. TuHeorem. If G is a compact semi-simple Lie group then every 
infinitesimal deformation of the action of G on a differentiable G-space is 
trivial. 


In [7] the authors prove a global form of this theorem; namely that if 
G is any compact Lie group (not necessarily semi-simple) then every deforma- 
tion of the action of G on a compact differentiable G-space is trivial. It does 
not seem that either of these theorems implies the other in any obvious way. 

Now let & be a finite dimensional subalgebra of the Lie algebra Y of 
all differentiable vector fields on a compact differentiable manifold 9. If V 
is a compact semi-simple Lie algebra (i.e. the Killing form is negative 
definite) then the simply connected Lie group @ with Lie algebra 9 isomorphic 
to & is compact and semi-simple. By Corollary 2 of Theorem XVIII of [6] 
an isomorphism of 9 onto ¢ is the infinitesimal generator of an action of ( 
on Applying 11.1 


11.2. THrorEemM. If ¥& is a compact semi-simple sub-algebra of the Lie 
algebra Y of differentiable vector fields on a compact differentiable manifold, 
then every derivation of £& into V is the restriction to £ of an inner deriva- 


tion of V. 


12. Lifting of group actions. In this section we assume that our 
compact Lie group G is connected and denote by G its simply connected 
covering group. We shall identify the Lie algebras of G and G under the 
isomorphism given by the differential of the covering homomorphism. We 
note that a differentiable G-space 9m is in a natural way a differentiable 
G-space and that the homomorphisms of § into the Lie algebra Y of differ- 
entiable vector fields on 9% which are the infinitesimal generators of the 
actions of G and G on 9 are the same. 

In general a homomorphism X > X* of Y into V is not the infinitesimal 
generator of an action of G or even G on 9 if In is not compact. However 
it is shown in [6, Theorem III, p. 95] that if each of the vector fields \* 
generates a global one-parameter group of diffeomorphism of 9m then Y¥ > \* 
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is the infinitesimal generator of a unique action of G on In. Now if X* 
does not generate a global one parameter group of diffeomorphism of 9m then 
(for example, see [6, p. 84 and p. 73]) there is an integral curve o of X* 
defined on an intevral [0,a) or (a,0] such that lim o(¢) =o (i.e. for each 


compact subset K of 9n o(t) ¢ K for ¢ sufficiently close to a). Suppose now 
Y is a vector field on a differentiable manifold 1, X a vector field on 
and f: — is a differentiable map such that df(Yp) = (under these 
circumstances we say, following Chevalley [1, p. 84] that Y and X are f- 
related). Then if o is an integral curve of Y foo is an integral curve of XY 
and it follows from the above remark that if Y generates a global one para- 
meter group of diffeomorphisms of 9 and if f is proper then Y generates a 
global one parameter group of diffeomorphisms of 7. 

Now let ¥ > X* be the infinitesimal generator of the action of G on a 
differentiable G-space 9m and let x be the projection of a differentiable fiber 
bundle B over 9, having compact fiber. Then 7 is proper and it follows 
that there is a one-to-one correspondence between actions of G on B for 
which is equivariant and homomorphisms 7: > X7 of into the Lie 
algebra of differentiable vector fields on 9m such that Y7 and X* are -r- 
related for all Y¢€ &. We now specialize further and assume that B is a 
principle-bundle with structural group a compact connected Lie group H 
and we write (h,b)—>hb for the action of H on B (this conflicts with the 
more customary usage in which the structural group acts on the right, but 
it is only necessary to define hb to be bh-"). 

For a diffeomorphism of B to be a bundle map, i.e. equivariant with 
respect to the action of H, means just that it commutes with each operation 
of H; hence if Z—> Z* is the homomorphism of the Lie algebra % of H into 
the differentiable vector fields on B which generates the action of H, then a 
one parameter group of diffeomorphism of B consists of bundle maps if and 
only if its infinitesimal generator Y satisfies [Y,Z*] 0 for all Z€ #. Thus 


12.1. THrorEM. Let X—X* be the infinitesimal generator of the 
action of G on a differentiable G-space In. Let H be a compact connected 
Ine group with Lie algsbra &, B a differentiable principle H-bundle over M 
with projection + and Z—»Z* the infinitesimal generator of the action of H 
on B. Then there is a one-to-one correspondence between actions of G on 
B equivariant with respect to such that each operation of G on B is a bundle 
equivalence, and homomorphisms 7: X > X7™ of & into the Lie algebra of 
differentiable vector fields on B such that 


(1) A7 and X* are z-related for all € § 
[X7,Z*] =0 for all XE ZE H. 
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A homomorphism +r: Y > X7 satisfying (1) and (2) will be called a 
lifting of 8 to B. A linear map 7: YX > X7 of § into the Lie algebra of 
differentiable vector fields on B which satisfies these conditions will be called 
a pseudo-lifting of § to B. To construct a pseudo-lifting of & to B it is 
only necessary to choose an H-invariant Riemannian-metric for B and let .\,7 
be the unique vector at b orthogonal to the fibre which projects onto \*,,,). 
It is easily checked that X7 is differentiable and by construction it is 7-related 
to X*. That X7 is H-invariant and hence commutes with Z* for all Z¢ & 
follows from the invariance of the metric. Liftings of § to B on the other 
hand need not always exist and one of the principle results of this section 
is that they in fact do exist (and are essentially unique) if H is a torus and 
G is semi-simple. In general we have the relation Z*,,—=dh(ad(h-")Z)*, 
for Z€ #, he H. We now make a final simplifying assumption, namely that 
H is a torus so that it follows from the above relation that Z* is an H- 
invariant vector field on B for all Z€ &#. A vector field Y on B is called 
vertical if 8r(Y) =O (i.e. Y is z-related to zero). Clearly for each 
pe Mm Z—>Z*|x*(p) is an isomorphism of & with the space of vertical H- 
invariant vector fields on z*(p). It follows that every vector field Y on B 
which is vertical and H-invariant is of the form b— (f(z(b)))*, for a 
uniquely determined function f: 9 — &, moreover Y is differentiable if and 
only if f is differentiable (the latter meaning of course that lof is differ- 
entiable for every linear functional / on ®). It follows that we may identify 
the space M of differentiable maps of 9 into # with the space of all vertical 
vector fields on B which are H-invariant (or, equivalently, which commute 
with Z* for all Z€ &). We note that since & is abelian any two elements 
of M, considered as vertical vector fields, commute. If Y is a vector field on 
In and m€ M then Ym is a well defined element of M; namely its value 
(Ym)(p) at p€ Mm is the unique element of # such that 1((¥m)(/) 
=Yp(lom) for each linear functional 1 on 9. Moreover by considering 
local product representations of B it is easily seen that if Y’ is a vector 
field on B z-related to Y then Ym=[Y’,m]. Clearly M becomes a 9: 
module if we define Ym = X*,, for X € ¥, m€ M. Moreover from the previous 
remark Ym = [X7,m] if is any pseudo-lifting of to B. If F 
is the tensor module of differentiable real valued functions on 9 and 
X1,° is a basis for then (f,,- -,fn)—>fi:Xit: (where 
the latter means the elment of M whose value at p is Sfi(p)Xj) is an 
isomorphism (as §-modules) of the n-fold direct sum of ¥ with itself and WV. 
It follows from the theorem of §9 that H*(%,M) =H,(%,M) and that if 
G is semi-simple then H!(%,M) = H?(%,M) =0. We collect these results as 
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12.2. THroremM. With the assumptions of 12.1 and the additional 
assumption that H is a torus let M be the linear space of differentiable maps 
of In into H. If we identify m€ M with the vector field on B whose value 
at b is (m(x(b))*), then this gives a linear isomorphism of M with the 
linear space of differentialble vertical vector fields on B which commute with Z* 
for all Z€ &. Considered as vertical vector fields on B any two elements of M 
commute. Moreover M is a §-module satisfying H*(%,M) =H*;(9,M) 
(and hence H'($,M) =H?(%,M)=0 if G is semi-simple) the module 
structure being characterized by the relation Xm =[X7,m] if r: XY X7 ts 
any pseudo-lifting of & to B. 


Continuing now with the same assumptions, with each pseudo-lifting 
7: ¥>X7 of & to B we associate a map c,; of & X § into vector fields on 
B by cr(X, = Y7] —[X, Y]7. Clearly c; is a measure of how much 
r fails to be a lifting, i.e. c-==0 if and only if 7 is a lifting. Now [X,Y]7 
is w-related to [X,Y ]* and by [1, p. 85] [X7,Y7] is w-related to [X*, Y*] 
=[X,Y]*. It follows that c;(X, Y) is z-related to 0, i.e. is vertical. Since 
and Y7] commute with Z* for all Z€ ([X, Y]7 by definition 
of a pseudo-lifting, [7,7] by the same plus the Jacobi identity) so does 
cr(X,¥), hence we can identify c;(X,Y) with an element of M. Moreover 
c, is clearly bilinear and skew-symmetric and hence an element of C?(%, 3/) 
If X,, X., X; belong to § then 


[X.", — [Xa Xa]? + Xs) 


Now [X,7, [X.7, X37]] satisfies the Jacobi identity, i.e. its sum with its 
two cyclic permutations is zero. Since 7 is linear so does [14,[X2, Xs] |’. 
Writing out the cyclic permutations of the above equation and summing we 
get an equation which gives dc;(X,, =0, hence c,€ We 
call c; the error cocycle of the pseudo-lifting +. 

Now let y€ C1(%,M), i.e. y is a linear map of § into M. Then it is 
clear that o: Y>A°=—X7+ y(X) is another pseudo-lifting of § to B 
and that conversely every pseudo lifting of & to B is of this form for a unique 
y€0*(%,M). We call y the difference cochain of « and 7. Recalling from 
12.2 that [y(X),y(V)] =0 for X¥, Ye 


= [X*, — —y((X, + —Fy(4) 
—c,(X,¥) + dy(X,Y) 
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or in words, the difference of the error cocycles co and c; is just the co- 
boundary of the difference cochain y—o—r. Thus the set of error cocycles 
associated with pseudo-liftings of 9 to B is an entire cohomology class 
wo € H?(%,M) which we call the obstruction to a lifting of & to B. By 
what we have seen above a lifting of & to B exists if and only if » contains 
the zero element, thus we have the only apparently tautological statement 


12.3. Tuerorem. I[f B is a principle torus bundle over a differentiable 
G-space In, then a lifting of 9 to B exists tf and only if the obstruction to 
a lifting of & to B vanishes. 


Since » € H?(9,M) which by 12.2 is zero if G is semi-simple. 


12.4. Corotiary. If G ts a compact semi-simple Lie group and B ts a 
principle torus bundle over a differentiable G-space In then there is a lifting 


of & to B. 


We now consider the uniqueness problem for liftings of § to B. 
Returning to our general situation let o and 7+ be two liftings of 9 to B. 
Their difference cochain y is clearly a one-cocycle. Conversely y€ Z'(%, M) 
and 7 is a lifting of 9 to B then so is o=r7r+y. Assume now that the 
obstruction cocycle w is zero so that a lifting + of & to B exists and assume 
also that H'(%,M) —0 so that by the above remark every lifting o of § 
to B is of the form ++ dm for some m€ M—C°(Y,M). More explicitly 
Xo = X7+ Xm + [Xm]. 

The most general bundle equivalence of B is of the form b—>f(x(b))b 
when f is a differentiable map of 9 into H. Since Exp: ®—H is the 
universal covering map of H it follows that every such f can be written in 
the form Expo m for some m€ M. Moreover by taking a local product repre- 
sentation of B it is easily seen that if h is the bundle equivalence given by 
b— (Exp(m(r(b)))b then = (X7+ Xm), i.e. = dhoXtoh, 
It follows that if (g,b) — gb is the action of G on B generated by 17 then the 
action generated by X% is (gb) >hgh-'(b). Since H'(9,M) =0 if G is 
semi-simple combining these remarks with 12.1 and 12.4 we have 


12.5. THEOREM. If G is a semi-simple compact Lie group, M a differ- 
entiable G-space, and B a torus bundle over In, then there is a differentiable 
action (g,b)—>gb of G on B which is equivariant with respect to the pro- 
jection of B on Mn and is such that each operation of G on B is a bundle map. 
Moreover this action is essentially unique in the sense that every other such 
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action of G on B is of the form (g,b) —>hgh-*b where h is a differentiable 
bundle equivalence of B. 


The latter theorem can be significantly generalized as follows 


12.6. THrorem. Let G be a simply connected compact Lie group, H 
a solvable, connected Lie group, In a differentiable G-space and B a differ- 
entiable principle H-bundle over Ir. Then there is a differentiable action of 
G on B such that the projection of B on In is equivariant and each operation 
of G on Bis a bundle map. Moreover this action is essentially unique in the 
sense that any other action of G on B with these properties is related by 
conjugation with a bundle equivalence as in 12.5. 


Proof. By induction on dim H. If dim H —1 then either H is a circle 
group and the theorem is a consequence of 12.5 or else H is isomorphic to 
the additive group of real numbers. In the latter case H is solid so that 
[8, p. 55] B is a product bundle so that existence of the required type of 
action of G on B is obvious. Uniqueness can be proved just as in 12.5. 
Now suppose dim H > 1 and that the theorem holds for all H of smaller 
dimension. Then H has a closed normal subgroup N such that both N and 
IT/N are conected solvable Lie groups of dimension smaller than that of H. 
Now B is a principle N-bundle over the orbit space B/N and B/N is a principle 
H/N bundle over and the composition of the projections B> B/N InN 
is just the bundle projection B— %n. By the induction hypothesis we can 
lift the action of G—first to B/N and then to B. Moreover any action of the 
apropriate sort on B induces one on B/N. Since the liftings from 9n to B/N 
and from B/N to B are essentially unique by the induction hypothesis, the 
same follows easily for the lifting from 9n to B. 
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AN INVARIANT CRITERION OF HYPOELLIPTICITY.* 


By Francois TREVES.+ 


Introduction. A differential operator P on a C® manifold Q is said to 
be hypoelliptic in © if, whatever be the distribution u on ©, u has to be a C® 
function in any open set in which Pu is a C® function. 

Several sufficient conditions of hypoellipticity have been given, from 
classical ones: the elliptic operators are hypoelliptic and so are the parabolic 
ones, to more recent (and weaker!) like the formal hypoellipticity, introduced 
by Lars Hérmander and Bernard Malgrange. Beside ellipticity, none of those 
sufficient conditions is invariant: they are properties of the expression of the 
operator in a system of coordinates which do not subsist when one changes 
coordinates. 

Since the definition of hypoellipticity is itself obviously invariant, it was 
natural to look for invariant conditions. In the present paper, we define a 
class of differential operators, which we call regular, by invariant a priori 
estimates, and we prove that a regular operator is hypoelliptic. 

This result can be considered only as a first step on a difficult road. For 
what one should wish, is to have properties of the expression of the differential 
operator in any system of local coordinates which insure hypoellipticity ; and 
that we do not provide. However, in the last section, we prove rapidly that 
any formally hypoelliptic operator is regular. 

The main step in our proof consists in deriving estimates about the com- 
mutator [P,p.*] of the differential operator P with Friedrichs mollifiers 
from “similar” estimates about the commutator [P,Q] of P with arbitrary 
differential operators Q@ (§6). This could be of interest in questions which 
do not bear any relation to hypoellipticity. 

Let us also mention that the reading of the article does not require any 
knowledge about distributions but the most superficial. 


1. Differential operators and their derivatives. Let Q be a C® mani- 
fold of dimension n=1. As usual C*(Q) will denote the vector space of C” 
complex valued functions on Q. A differential operator P on Q is a linear 
map of C*(Q) into itself which satisfies the following conditions: 


* Received April 27, 1961. 
¥ The author is a Fellow of the Sloan Foundation. 
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(DO;); For every compact subset K of © there is an integer Wx = 0 such 
that if all the derivatives of 6€ C*(Q) of order = Mx converge to 
zero uniformly on K, P/—0 uniformly on K. 

(D011) The support of P¢ is contained in the support of ¢.* 


We denote by » the variable point in Q. If T is a letter, ¢ any element 
of C*(Q), it follows from (DO;) that, on every compact subset of 0, 


e- P ) 


is a polynomial in 7 of degree m(¢,w) depending on ¢ and on the point o; 
m(w) =sup m(¢,w) is the order of P at. For any subset A of Q, - m(w) 


is the order of P in A; observe that it could be infinite (it is eietainiy finite 
when A is compact). ‘ei this article we shall not make the assumption that 
m(w) is independent of wo. 

Let 9’ be an open subset of © covered by local coordinats (21,- - -,#”). 


In Q’ we may look at the partial differentiations ar ie a and their 


products which we shall write 
@ 
me (i Qin 


the a; are integers and a;= 7; we set a= -,a,) and 7, which is 
called the length of a, will be denoted by |«|. If ¢€C%*(0’), then for rE 0’ 


= Aa(x) Dad (2) ; 


here can be regarded as the vector -,x"); the coefficients da(z) 
are C* functions of x; for every « the summation is performed over a finite 
set of multi-indices «, actually over those a satisfying |«#|=m(a). The 
“ differential operator” } Aa(x)Da is the expression of P in the coordinates 
(a1,- - -,a"); we shall usually denote it by P(a,D). 

To the expression P(x, D) we may associate the polynomial (with coeftfi- 
cients in C® (’) ) 


P(x, €) => Aa(@)éa, fa = * *§a,3 
and deg P(z,€é) =m(ar). If +,Bs), 1S we set: 


1Mr. J. Peetre has proved that (DO,;) implies (DO,). See J. Peetre, “ Une carac- 
térisation abstraite des opérateurs différentiels,’ Math, Scand., vol. 7 (1959), p. 211. 


Pip 
(2, 
-( 
PC 
x 
€) 
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If Q is another differential operator in 0’, these notations allow us to express 
the commutator [P,Q] in the coordinates -,2": 


—1)! 
1.0) D) Poy (0D). 

The proof of (1.0) is of course elementary. Let us derive from it two 
important particular cases. First we take Q of order zero; Q is then simply 
the multiplication by a C* function y (on Q’). Then (see Hérmander [1]) : 


1 Oy 
P,Q] P‘8) (x, D 
We use here, for simplicity, the notation 


0 
we have (im) 


Second, let us assume that the expression of Q in the coordinates 2/ is a 


polynomial in with constant coefficients. Then: 
—1) (él 


We want to derive from (1.1) and (1.2) two other formulas which will 
be useful later. 

For that, we need to put an order relation (actually, a pre-order relation) 
on the multi-indices B. Let - -,a,), B= -,Bs), 
be two such multi-indices. We say that B= if, for every 
p=1,:--+,n, the number of 6; equal to p is = number of a; equal to p. 
If a= and Ba, we write ~~: a and B are then identical modulo a 
permutation. If 8 = without «= we write B With those notations: 


BSa | 
Also: 
0 


1 
Therefore: 


~ . 
a 
! 
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Similarly, by applying (1.2) this time: 


— 1) 
(1.4) (—1)!l Pee) (2,D) = [P,Da]— (D,) (2,D). 
o<pca | BI! 
2. The norms || ||, We consider now the n-dimensional real space R” 
with Cartesian coordinates 21,- --,#"; we set 


dz = dz'- - -dx". Let us denote by Co” (R") the subset of C*(R") consisting 
of functions with compact support. We define the Fourier transform of 
p€ (R") as 


where belongs to the dual R, of R” and -+2"&. The 
reciprocity formula holds: 


a. 
Rn 
We set |é|=(/& |? +: --+]é,|*)4. Let s be any real number. We set: 


The norms || ||, are invariant in the following sense: Let » be an auto- 
morphism of the C* structure of 2”; ¢—> $7 is a one-to-one mapping of 
Cy” (R") onto itself. For every compact subset K of R” there is a constant 
Cr such that, if Co*(R") and if support of dC K, 


loons SCu 
(see Peetre [1], p. 18). 


Let us go back to the manifold ©. For any open subset 9’ of Q, 
we denote by C,*(Q’) the C® functions in Q whose supports are compact 
and contained in 9’. Assume that ©’ is covered by local coordinates 
x,- --,a" and let J be the diffeomorphism they define from 9’ onto an 
open subset O’ of R"; then ¢—>¢0J- is a one-to-one linear map of 
(Q’) onto We may define the norm || of Co” (’) 
by equating it to || Of course | ¢ ||, depends on the coordinates 
ai; but if we change coordinates and denote by ||| ¢|||, the new corres- 
ponding norm, to every compact subset A of ’ corresponds a constant 


such that 


provided that the support of ¢ lies in K. This is a consequence of the invar- 
iance of the norms || ||, in R" we have indicated above. 
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Let now K be an arbitrary compact set in 2. We may construct a 
partition of unity in Co*(Q) over some neighborhood of K such 
that the support of each a; lies in some local map ’;. In ’; we may define 
the norms || ||, and then, for ¢6€ C*(Q) with support in K, we may set; 


If one changes the partition of unity or the local coordinates, one gets an 
equivalent norm. Anyway, let Cx” be the set of ¢€ C,*(@) having their 
support in K. The completion of Cx® for the norm || ¢ ||; is denoted by H*x 5 
H®x is a Hilbert space (but the Hilbert structure depends on the partition 
of unity {a;} and on the local coordinates”). 

Let us assume that we have used the same partition of unity and the same 
local coordinates in defining H*,; whatever be s real. Then we can state the 
following properties : 


1) The identity mapping of Cx® can be extended, when s=t, as a 
one-to-one continuous linear map H*x— H'x, in fact of norm <1, which we 
shall call the natural embedding of H*x into H'*r. 


LemMA 1. If s>t and s=0, the norm of the natural embedding 
H*;— Htx tends to zero when the diameter of K tends to zero. 
For the proof, see e.g. Malgrange [1] (Lemma I. 2.3). 


2) Let Q be a differential operator defined and of order » = 0 on some 
neighborhood of K. Then, for 


|| Qo S Ax (s,Q) | > 


where Ax(s,Q) < +00 depends on s, Q, K but not on ¢. Hence ¢>Qg¢, 
which maps Cx* into itself, expands as a bounded linear map H**4#_— Hx. 
The case »=0 is important: Q is then the multiplication by a C® function 
y on some neighborhood of K and, for ¢€ Cx®, 


All this is valid whatever be s real. 

When Qf", we may define, for s real, the completion H* of C)* 
= 0,* (R") for the norm || ¢ We shall make use of different properties 
of the spaces H*: 


i) forts, H* C H'‘; H® is dense in H*; the norm of the embedding 
H'—> Ht is =1; 


* However, as a topological vector space, H*, is invariant. 


a 
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ii) for ¢,~€ Co”, the bilinear functional 


fo ay (z)ae 
can be extended to H* & H-* and turns those spaces into the duals 
of each other. 

Now for a few words about distributions in Q. The elements of H*x 
can be canonically regarded as such distributions, whose supports, morevoer, 
lie in K. If wu is a distribution with compact support K,, we shall use the 
notation wé€ H* to indicate that w¢€ H*x whatever be K compact, K D Ky. 
Similarly, we shall use expressions like: a set of distributions wu bounded, 
compact, etc. in H*; a net of distributions converging in H*; etc. This 
applies only, when © is an arbitrary manifold, in our set up, to sets or nets 
of distributions having all their support in some fixed compact subset K of 
Q and means tha; the properties in question really occur in Hx. 

Given any distribution wu on © and any g€ C,*(Q), there is a real 
number s such that ¢u€ H*. In general, s depends on ¢. If s can be taken 
independently of ¢, we say that wé€ H*%.(Q). In other words, given any 
distribution v in Q and any open set 9 C Q, O compact, there is a real q such 
that the restriction of v 19 0’ belongs to If w€ H%1-(Q) whatever 
be s real, w is a C® function on Q. 

Let us finally add that all the differential operators will be assumed, 
in the sequel, to act in the sense of distributions. 


3. Some lemmas. Let now p be a function on Re, p(x) dz—1. 
R* 
For t > 0, we set: The Fourier transform of p; is 


2. ds 
(pr)*(y) = jn = (ty). 


LemMMA 2. For every real number p’ there is a number »=0 such that, 
for all multi-indices B 1S for such 
that |B| =p, we have, for all numbers 0<t <1, 

|| pe SC(| 
where C(|B|) < +00 depends only on p and | Bj. 

Let =sup(p’,0). 


SF y | (ty) |? dy 


— #161 (1+ | y | (Dpp) (ty) |? dy 
| ty | (Dpp) (ty) |? dy 
— | Dgp(é) |? dé 
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and therefore any has the desired property. 
For y€ Co”, we set: 
¥ sup (1+ |é 
f€Rn 
If Co* also, then: 
(3.1) 


2)o/2 | J(é)|, o real. 


whatever be the real numbers o’, o” such that o’ +0” =o. 


LEMMA 3. 


Let B be a multi-inder: B= +; Br) 


for 
Whatever be 0<t< 1, 


where C(| B|) < depends only on p and 
We have: 


sup 


ye 


2) | (Dg) wor (y) | 
= #lél(1+ | y|?)!8l/? | (Dep) (ty)| 
Rn 
= sup (1+4+|ty (Dep) (ty) | 
sup (1+ Ol < 


2) [Bl /2 


ye Rn 
€Rn 


(the Fourier transform of a C,* function decreases, as well as each one of its 
1 
derivatives, when | £|—>- oo, faster than any power of Te) 


COROLLARY. 


GE Co, 


Let o be any real number. Whatever be 0<t<1 and 


| lo SC(|B) il 
Immediate consequence of Formula (3.1) and Lemma 3. 


Lemma 4. Let O be an open set in Rn, R(x;y) a C® function in 


O X O whose derivatives, up to the order k —1 included (k=1), vanish on 
the diagonal xy. Then one may write 


where the Ra(x,y) are C® functions in OX O. 


We recall that (2—y)* = — y%) - (a%r— where lS 
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for j=1,---,r; a=(a,,:--+,a,). It is enough to prove the Lemma for 
k ==1; by iteration, one gets it then for arbitrary k. Let us therefore assume 
that R(z;y) =0 if For j=—1,- -,n, let us set: 


Since R(y?,- - -,y";y',- -,y") =0, we have 


Biew) => 


We have to show that the R;(x;y) are C® inO XO. But it is enough to 
show it when » — 1, in which case the result is trivial. 
Let be a function on K We set: 


(2) = f (a, uy (udu, 

where ¢,y€ 

LeMMA 5. Let o, r be two real numbers. For every $,y€ Co”, 

C(K 30,7) is a constant < +00 depending on K, «, r but not on ¢, wy. 

Of course, Co® =C,*(R"). We set s=o-+r. 

We may write K(a,y) = ferincow urd K (y, v)dydv, where K(y,v) is 
a function belonging to dy,,.* In particular, whatever be the integer k = 0 
the function (1+ ]|y|+]|v|)*K(y,v) is square integrable in R,”" X R,". 


Let us set 


H (2, v) = — u)w(u) du. 
We have: 
(3.2) (x) = (K(y,v)H (2, y, v) dydv. 


It is obvious that H(z,y,v) is C® in 2, y, v, and, regarded as a function 
of x, has its support in a fixed compact set independent of y, v. On the other 
hand, H(z, y,v) is bounded in R*". From this follows that the Fourier trans- 
form of the right hand side of (3.2) equals 


fE(y,v)H (é,y, v) dyde, 


* & is the space of C® functions of w € R™ decreasing at infinity, as well as each 
one of their derivatives, faster than any power of 1/| w |. 
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where H (é,y,v) is the Fourier transform of H(z,y,v) with respect to 2. 
We have then, in view of Schwarz inequality, 


(3.3) 1+] OPS 0)? dyao) 


where & is any integer =0. We shall show now that under the hypotheses 
of Lemma 5, there is an integer V = 0 such that, for all y,v€ Ry, 


Choosing then * > N + 2n and integrating with respect to € both sides of 
(3.3), we obtain the lemma. : 

We have H(é,y,v) =¢(€—y)9(€—y—v). Let us now use the fact 
that for every real number r there is a finite constant C, such that for all 
€ Rn, 

We get: 
(1+ | (é&y,0) +[y fy +o 


By integrating the two sides of this inequality, we get 


+ | y | + v |) 


2 


At that point it should be noted that since K(x,w) has its support in a fixed 
compact subset of R,” X Ry", we may as well assume that the support of y 
lies in a fixed compact set K. Let a€ (o*(R"), a==1 on K. We have 
& and therefore 


(1+ | SCF A+] | 
X (1+ | | | S Co 


by applying Schwarz inequality. This concludes the proof of Lemma 5. 


4. Statement of the main result. Let © be an n-dimensional C* 
manifold. 

In order to abbreviate the forthcoming statements, we shall constantly 
make use of the following convention. 

Let R, S be two differential operators defined near a point of Q (i.e. 
on some open neighborhood of w). We shall make statements of the following 
kind: 


6 


| 
| 
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The inequality || R¢ C || Sd ||, ts valid at o. 


The meaning of it is the following: 

Let 0’ be an open neighborhood of » on which both R and S are defined. 

i) There is an open neighborhood U C 0’, depending on R, S, r, s and 
the local coordinates near w used to define the norms || ||, 

ii) there is a finite constant C, depending on R, S, r, s and the local 
coordinates, such that, for every ¢€ Co*(Q) with support in U, 


|| R¢ |, SC | 
Of course C is independent of ¢; we shall handle C as a kind of general symbol. 


In order to state the main result of this article, we shall introduce the 
following definition : 

Let »€ Q, P be a differential operator defined on 2. We shall always 
denote by m the order of P on some open subset of © containing w. 

Let o, a, b be real numbers. Let us consider the following properties 
which may be valid at the point of Q: 


(Ac) Il S C | Po Vo. 


(Bc) Whatever be the differential operator L defined and of order »=1 
near w, 


| [P,L Ip = C | Po 


DEFINITION 1. We say that P is regular at wo tf there are numbers oy 
real, a real, b >0 such that (Ac), (Bo) hold at w for every = ap. 

We say that P is regular in some open subset Q, of © if P is regular at 
every point » of Q (ao, a, b will then generally depend on the point w). 


Remark 4.1. If (Ac) is valid at o, a must necessarily be =m; a=m 
if and only if P is elliptic in some neighborhood of wo. 

If (Bc) holds at o, b must be =1 and b =1 if and only if P is elliptic 
near w. 

Our main theorem will then be: 


THEOREM 1. If P is regular in Q, P is hypoelliptic in Q. 
We shall prove a slightly more precise statement: 


THEOREM 1’. Let P be regular in Q; let Q, be an open subset of Q with 
compact closure. There is a real number a, such that, whatever be s real 
and the distribution u on Q, 


Pu € H%-.(Q,) implies u€ 


| 
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5. First consequences of (Ac), (Bo). Let us introduce the following 
property, which may be valid at w€ Q: 
(Bs)u* Whatever be theXx SM differential operators L,,- - -,L) of respec- 
tive orders dyS1,- -,d, near o, 
with d=d,+:--+d). 
LemMMA 6. Assume that (B;) holds at o, with b=0, for s=o, o+1, 
-ssotM (M21). Then (Bo.m)u* holds at o. 
Let us set Ry —[---[P,L,],---L)]. We are going to show that 


(5.1) || Rad SC || Po at o 
for s=o+A,o0+A41,:--:,o+M. For \=M, this will be exactly what 


we want. 

We reason by induction on A. When A=1, (5.1) is a trivial conse- 
quence of the validity of (B;) for s=o,0+1,---:,o+M. We assume 
X}=2 and that (5.1) is valid up to \—1. We have R,—[R).,,L,] and 
therefore, at w, setting 


| R\¢ R)_,Ly¢ ils d)\—d’+b | [)\R)_.¢ d’+b-dy 
S || || + C || || 


Assume that s belongs to the set of numbers {0 +A,---,o-+M}. Then s 
and s—d) (since d,=1) both belong to the set 


{o 
hence at w: 
| Ils d)\-d'+b | PL)¢ Ils 


R, Is. d’+b = C Po lle. 
Since b= 0, 
|| PLy¢ S || LyP + || LPs La] ¢ 
SC || Po + LPs Lx] ¢ 
But certainly s€ {o,- -,o-+ M}; hence at 


This concludes the proof. 


Lemma 7. Assume that (As) holds ato. Let M, be the smallest integer 
=2m—a+b. If (Bz)u,* holds at o, (Bs)y* holds at w whatever be the 
integer M. 


| 
| 
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We keep the same notations as in the proof of Lemma 6. Let 2*,: - -,a” 
be local coordinates near w. Let P(z,D), Ry(2,D) be the samen expres- 
sions of P and R) in those coordinates. We may write 


Ry (a2, D) => Ry, (x, D) Pip) (2, D), 


where the R),a*(z,D) are differential operators depending on L,,- - -, Ky. 
Observe that in the summation, because of the definition of R,, we will always 
have |a| =AaA—d, 

Assume A= M =2m—a-+b. Then we must have d= m—a-+b or 
else Ry =0 near. For if d< m—a+b, |a|=A—d>~m and =0 
near w. 


If d=m—a-+b, d+m+b = | 


Observe on the other hand that the order of R, near w is = m whatever 


be A. Hence at wo 


and by combining those two inequalities, we get at w 
|| Rag SC || 
Then the Lemma is simply a consequence of (A,). 
If (Bs)u* holds at » whatever be M, we say that (B,)* holds at o. 


Lemma 8. Property (B,)* is valid at w if and only if the following 
property is valid at w: 
(C,) Whatever be the differential operator Q defined and of order w near o, 


| [P, Ql¢ Il = C | P¢ Ils. 
1) (Be) => (Cz) 


Let us use again the local coordinates x/; let Q(z, D) be the expression 
of Q in those coordinates. According to Formula (1.0) we have at o: 


SC | Poy (a, D)¢ 


since the order of Qig)‘” (a, D) near w is Sp—|y|. 


Observe that if 1SASn, 1d’ Sn, 
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and therefore P,.) ‘®)(2,D) is the expression in the coordinates 2/ of 


a... 


assuming that B= °;Br); y=(y' ys). Then, from (B,)*, it 
follows immediately that, at w 


| Pion ® (2, SC || Po 


2) => (B,)* 
It is easily seen that [- - -[P,L,],- - -L,] is a sum of terms of the 


form 

where *,Jg) is simply a permutation of (1,2,---,A). We 
have at w: 


| L;, -L,[P, L;,- 
= C | P, L;, L;,\¢ Po ll ss 


where Lj,: --Lj, is precisely a 
differential operator of order d° near w. 

Corotuary. Lei z',:--,a" be local coordinates near wo, P(a,D) the 
expression of P in those coordinates. 

If (C.) is true at w, then whatever be the multi-indices a, B,|«|+ |B | 
> 0, 


| Pia) (2, SC | Po 
holds at w. 
The important result in this section is the following: 


LemMA 9. Assume that (Ac), (Bo) hold at w, with b=0, for every 
Let M, be the smallest integer = 2m—a+b. Let + Mo. 
Then (C,) holds at w for every s= 8». 


Lemma 6 tells us that (Bo), valid at » for oo’ So-+ Mo, implies 
(Bosa,) ato* at ow. 


Lemma 7 tells us that (Bo.m,),* ==> (Bo.m,)* provided that (Ac.m,) 
is true at w. 


Lemma 8 state that (Bo.m,)* is equivalent with (Co.m,). 


Q. E. D. 


657 
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Remark 5.1. From now on, instead of working on the basis that (Ac) 
and (Ba) are valid at » for o=a», we shall assume that (A,) and (C;) are 
valid at w for s= Sp. 

We conclude this section with a trivial result that is going to be needed 


later on. 


Lemma 10. Assume that (Cs) holds at w. Let Q’ be some open set 
containing w. There is an open neighborhood Q,” C of w such that for every 


we and every Co” (2s), 
¥]¢ SC || Po 


This means that the inequality in the Lemma is valid for ¢€ C,* (Q) 
having its support in a neighborhood of » independent of y. 


Proof. Let x1,---,x2" be a system of coordinates covering an open 
neighborhood ©,’ C 0’ of w; let P(z,D) be the expression of P in those 
coordinates. If Cy” (Q,’), we have, according to Formula (1.1): 


¥]¢ S || P (a, 


(in fact, we have to assume something like 9,’ compact, Q,’ C 9’). For every 
multi-index 8, | 8| > 0, we have according to the Corollary of Lemma 8, 


|| P (2, D) > SC || Po 


for ¢ € Co” (Q’(g)), where is a suitable neighborhood of w. Observe that 
(x, D) =0 near w for | > m. We may take then 
0<|B|=m 
6. Properties (A,), (C,) and Friedrichs mollifiers. We choose local 
coordinates z*,- - -,a" near w; we shall always assume that =: : - 
=w2z"(w) =. The norms || ||; are computed by means of those coordinates ; 
P(x, D) will be the expression of P in the a/, 


Let p be a function on Re, For t>0, we set: 


pe(@) =t"p(a/t). 


Let J be the homeomorphism ’— (2'(o’),- - -,a"(w’)) of a neighborhood 
Q’ of » onto an open neighborhood O’ of the origin 0€ R"; J(w) =0. The 
map ¢—>¢0°J7 is one-to-one from (Q’) onto C,*(0’). Let 2” CO be 
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open, Q” compact, 2” CO’. We may find t) > 0 small enough so that tS ty 
implies, for ¢€ Co” (”), that the support of 


[ota (oI) (a) = da! 


lies in O’. Therefore the support of [p:.(¢°J-?) |] oJ will lie in 9”. 
We still denote by the operator (Q”) > Cy” (’) 


[pra(pod| oJ. 


| This operator p;, can be extended as a linear operator from the space of 
distributions in ©, with compact support contained in 0”, into C)°(’). 
In particular (and that is what we are going to need later on), if we H*x, 
K compact, K CQ”, p:,u is a Co* function (with support in 0’). Moreover, 
when ¢ > 0 tends to 0, wu in the sense of 

From now on we assume that the number ¢) defined above equals 1 
We may always be in that case by just replacing p(x) by to"p(a/t). 


Lemma 11. Assume that (Ac), (Co) hold at wo. Then 


holds at w independently of 0<t<1. 


By “independently of ¢” we mean the following: 1) the constant C < +00 
does not depend on ¢; ii) we may take an open neighborhood N of «, indepen- 
dent of ¢, such that the inequality is true for every 6€ C)*(N). Here, of 
course, both C and N depend (in general) on the function p. 


Proof. According to Lemma 4, we may write, for some integer p= 1, 


Pip) (ts) + (2 —u) De) 


where the Rg(z,u,D,) are differential operators on O’. The coefficients of 
are C* functions of € O’ O’; the order of Rg(z, u, Dz) 
is =m. 


If O” =image of 2” under J and we Cy*(O0”), we have: 
D) (pes = P(r, Dz) f (u) du 


i 
4 
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We have taken into account the fact that 


Q (De) = (u) du, 


where Q(D) stands for any polynomial (with constant coefficients!) in 


da?” 
According to the Corollary of Lemma 3, 


independently of ¢. Let us apply then the corollary of Lemma 8. For 
|8| > 0, we have at o: 


Pia) (2, D)¢ SC || PS 
where ¢=yoJ.® We are therefore left with the estimation of the norm 
| 9 llox» of the function 
g(t) => f u) 8p: (a —u) Rg (a, u, Dy) p(w) du, 


which is a sum of terms of the form 


(x) = f Fp; (a — Rg* (a, u) (Daw) du. 


There is a compact set H C R", such that O” + H C O’ and that the supports, 
in Rk," R,", of the functions p:(~—wu) (Dep) (u) (whatever be « and 
0<t<1) are contained in (O” +H) xO”. Hence we may assume that 
the functions Rg*(z,u) have their supports contained in a fixed compact 
subset of BR," K R,”". This allows to apply Lemma 5: 


9* S C || pt || Dow 
where p’ is any real number (of course, C depends on p’). On the other hand, 
|| Day SC || 
We choose p’ = m—a-+b. According to Property (Ac): 


= SC || Po |lo at o, 
where ¢=yodJd. 


° Ps) (a,D) is a differential operator on the open subset O of R*. Through the diffeo- 
morphism J, it defines a differential operator on &’ C 2 whose expression, in the coordi- 
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Ultimately, we must show that we can choose the integer » in such a 
way that the numbers || ||, |8|— , are bounded independently of 
This follows from Lemma 2. 


Remark 6.1. No assumption whatsoever has been made upon the sign 
of b: we did not need 6 > 0 or even b= 0; b could be any real number. 


7. The last lemma. In this section and the next one, we assume that 
P has properties (A,) and (C;) at w, with 6 >0 and for every s=sq (8 
being some real number). We recall that those properties are: 


(As) va S C |] Po 


(Cs) Whatever be the differential operator Q of order p near o, 


It is essential that a real and b > 0 should be independent of s = 5. 


The three numbers so, a, b give, in some sense, a “measure” of the 
regularity of P near w. They will serve to determine the number a, of 
Theorem 1’ and some other regularity properties of P—as we are going to 
see. In view of those properties, let us say that a and b should be as large 
as possible while s) should be as small as possible. We recall however that 
a cannot exceed m and 6b cannot exceed 1 (Remark 4.1). We shall actually 
use those facts. 

Anyway, in order to abbreviate the forthcoming statements, we shall say 
that, under our assumption, P 1s of type (8, a,b) at o. 

Let z1,- - -,2” be local coordinates near w. We introduce the operator 


A 
U,= (1—7a)?; p: integer = 0. 
T 
A= U, is defined on some open set containing o. 


Lemma 12. If P is regular at o, so is PU,. More precisely, if P ts 
of type (so,a,6), PU, is of type (s*—2p+b,a+2p,b) at o, with 
So* = sup (50, 0). 


Let We have at o: 


|| PU = || Up || — || Un] 
= || Po —C || Po Ilo. 
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Since ¢=0, b >0, we may apply Lemma 1: if the support of ¢ lies in a 
suitable neighborhood of o, 


C || Po S |] Pod lows. 


Therefore, at w: 
(7.1) || SC || PU pd || o-2ps0- 


Let us prove first (A,) for PU, and s=sot—2p+b (moreover, with 
a-+ 2 instead of a). We have at w, ior c= 8: 


| = C | Po (since b 0). 
By applying (7.1) and setting o—s + 2p—b, we get at o: 


S C |] PU po 


which is what we wanted. 
Let us prove now (C,;) for PU, and s=s.*—2p+b. Let @ be any 
differential operator of order » near w. We have 


Note that [U,,Q] is a differential operator of order [2p + p—1 near w. 
Hence we have at o: 


in order to obtain the last term of the left hand side, we have used the fact 
that b= 1. 

Let us take s=o—2p+b, o=s8,*. The order of U,Q near w being 
2p +p, we have at w, because of (Cc) for P: 


LP; UrQ) || +0 S C || 
On the other hand, at o: 
LP; Up]¢ S C || Po 
We have therefore, at o: 
| [PU,, = C P¢ 


Then (7.1) gives the result. 


Q. E. D. 


wi 
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Let us now take mollifiers p; as in §6. We shall consider the following 


property of a distribution wu in Q: 


(R)o There is an open neighborhood No of » such that the following is true, 
for (Na): 


i) P(du)€ 
ii) when t->0, Pl[ptx(du)] > P(du) weakly in 


Lemma 13. Let P be regular of type (8,a,b0) ato. If we H**™,.(Q) 
and Pu€ H*;.(Q), then holds. 


Since H**™,,.(Q), (Rs, is trivially true. Let us assume that (PR), 
holds for some «= So. Of course we assume s >o. 

Let 0”o be the open neighborhood of » considered in Lemma 10 (where 
we take QO’). Let us set 


= N No. 


Let U, V be two open neighborhoods of », with the following properties: 
U, V are compact, U CV,V CN’o. If $1€ Co*(V), we may find a number 
ty > 0 such that, for tS to, prx (div) has its support in N’o. Let (U) 
and let us assume that ¢,—1 on a neighborhood of U. We have: 


P (gu) = + [P, 
We have then, for ¢ < fo, 
ILP, } SC || Plptx } 


According to (f)> the right hand side is uniformly bounded with respect to 
t—0. We use now the fact that closed bounded set are weakly compact in 
the functions [P, 6] {p:(¢:u) } must weakly converge in H**? as t> 0. 
Their limit can only be [P,¢](¢:1u), which therefore belongs to H**?. 

Let c be a number Sincec+ess, dPuc 
since c=), [P,¢] (¢iu) € hence P(du) € 

This being done, we apply Lemma 11. There is an open neighborhood 
W of w and a constant C < +0, both independent of tt, (t)>0 suffi- 
ciently small), such that, for y€ Co* with support in W, 


P (pes) lore S LPs pea ly llove + || 
S pre + || ptaP  llore 
| Py Ilo | pty Py ll ore 


664 FRANCOIS TREVES. 


Let U’ be an open neighborhood of w, such that U’ is compact and 
U’ CW. Let us set Nop =UNU’. If 6€Co*(Q) has its support in 
Nasco W =pex(¢u) has its support in W as soon ase > 0 is small enough. On 
the other hand, P(g¢u) € 


When «— 0, 
i) ptgPy— (du) in trivially ; 
li) > in trivially ; 


iii) Py—P(¢u) weakly in H? because of (R)c. 
At the limit, we have: 
P(pre (PU) ) SC || P(pu) + || peal (PX) 


When 0, (du) > P(du) in since P(du) € We derive 
from there that P(p:,(¢w)) remains bounded in H*%*¢, therefore converges 
weakly in H**¢, necessarily to P(@u). We have thus proved that whatever be 
S Sos, for cZ0, cS cSs—co. Take an integer 
large enough so that c= (s—s.)/v=b. We have proved that for every 
p=0,° (RB) => For »=v—1, we get (R),. 


Lemma 14. Same hypotheses as in Lemma 13 for P, u, Pu. Then o 
has an open neighborhood U such that ¢ € Co” (U) implies gu € H**4, 


Let us assume s = &o, for if s were < 8), s+ a would be < s) + m (since 
always a= m) and since u€ H**",,.(Q), there would be nothing to prove. 


We may first choose U in such a way that for ¢ sufficiently small, 
pte (GU) S C || P (pte (pw) 


If we take U C N, (neighborhood of », used in Property (R),), we see that 
the right hand side is uniformly bounded with respect to ¢. We derive from 
there that p:.(¢u) converges weakly in H*** as t—>0, necessarily to gu. 


8. End of the proof of Theorem 1’. We make the assumption that 
Pu € H%io-(Q) (u: distribution on 2). It is enough to prove the following: 
let » be an arbitrary point of Q and assume that P is of type (so(w), a(w), b(w)) 


. 


nd 
in 
Mn 


p 
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at w (according to Remark 5.1). Then there is an open neighborhood U (w) 
of w such that 
C,° (0 (w)) => 


For let then Q, be an open subset of ©, with compact closure. We may cover 
Q, with a finite number of such open sets U(o,),: - -,U(o,) and take 


a, =inf(a(o,),° +,a(o,;)). 


In other words, we must extend the conclusion of Lemma 14 to an arbitrary 
distribution wu in Q. But if Q, is open, 2, compact, 0, C Q., there is a real 
number g such that the restriction of wu to 2, belongs to Hig4(Q.). We may 
therefore assume that u€ H%.(Q). 
Let w be any point of 0; let a',- - -,2" be local coordinates near w and 
let us introduce the operator 
A 


U,=(1 ar ae p: integer = 0. 
Tv 


We assume that P is of order (8 ,a,b) at w. Let us choose p large enough 
so that g++ 2p=s.*+m-+), with so*=sup(s,0). There is a distribution 
v on some neighbrohood of » such that U,v in Q’. And 
C (0). 

We have PU pv € We know that PU, is of type (so* 2p + ), 
a+2p,b) (Lemma 12). Let us set ro —=s)t—2p-+b. We see that 
v€ Hor(m+2p),,.(Q’) and m- 2p is precisely the order of PU, near w. We 
therefore apply Lemma 14, with PU, instead of P, 0’ instead of Q, 7) instead 
of so, ete. There is an open neighborhood of such that (Q*) 
=> go € 

Let ¢; € Co” (Q*) be identically one on some epen neighborhood 2** C 0* 
of w; € Let Co? (Q**). We have: 


gu = pU pv = 
But Up(d.v) € 
9. Regularity of formally hypoelliptic operators. Let P(D) be a 


differential operator with constant coefficients on R". Hérmander has proved 
(in [1]) that P(D) is hypoelliptic if and only if there is a number d>0 
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such that, for every multi-index a= (1SajSn), 21, and 
for every €€ Ry, 
(9.1) (1+ | P (€)| SC(1+ | P(é))) 


where C < +0 is independent of é. 


Let 2, be an open subset of a C* manifold Q covered by local coordinates 
z,---+,a*. A differential operator P on Q is said to be formally hypoelliptic 
in the local map (,,2',- - -,2”") if its expression P(a, D) in the coordinates 
xj satisfies the two following conditions: 


(FHE), For every x €Q,, the tangent differential operator P (2p, D) 
(which has constant coefficients) is hypoelliptic. 


(FHE): Whatever be the points v’,x2”€Q,, there is a constant c>0 
(depending on 2’,x”) such that, for every €€ Ry, 


“= 


< 1+(|P(#,é)| —1 


Several proofs of the hypoellipiticity of the formally hypoelliptic operators 
have been given (first by Hérmander [2] and Malgrange [1]; see also Peetre 
[1], Treves [1]). We shall indicate briefly the demonstration of the following 
fact: 


THEOREM 2. Let us assume that each point of Q belongs to a lecal map 
in which P 1s formally hypoelliptic. Then P is regular in Q. 


Since the question is of local nature, we may reason in a neighborhood 
of the origin in R" (coordinates: z',- - -,a"). We may write 


P(x, D) =P(0,D) + 


where, for every j=1,: - -, N, aj(x) is a C® function in a neighborhood of 0, 
a;(0) 0, and P;(D) is hypoelliptic and satisfies 


| Ra. 


By hypothesis P(0,D) is hypoelliptic. Now, whatever be s real, there is an 


| 
| | 


and 


tes 
tic 
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open neighborhood O,; of 0 such that ¢€ C)>*(O;) implies, whatever be the 
multi-index « of length | «| =1, 


(9.2) | P (0, D) lleva, S C || P(0, D)¢ 

(9.3) | Pi (D)¢ lleva, SC || ls SC’ || P(0, D)¢ |e, 

where d, > 0, d; >0. Furthermore if s= 0, we will have, for those ¢, 
(9.4) || P(0, ||s SC || Po 


For the proofs of these statements, see Malgrange [1]. 


There is a multi-index « such that P‘®(0,é) is a constant 40. If the 
order of P(0,D) is 21 (which we may of course assume), | «| =1 and by 
combining (9.2) and (9.4), we get, at the origin and for s=0, 


Il SC || Po 


This proves (A,) for s=20 and a= d). 


In order to prove (B;), it is enough to consider the case where L is of 
order » 0, i.e. E is the multiplication by a C* function y defined near o, 


and the case where L = for a certain A, 1 [A= 
We have 


De N 
[Pv] = (P@ (0, D) + Saj(2) Pi (D)}. 
laj>o| j=l 
We may apply (9.2), (9.3). We get, at the origin 
I[P.¥]¢ | P(0,D)¢ 
with d=inf(do, d,,° - -, dy). 


Then (9.4) yields, for s=0 


¥1¢ S Cll Po ||. 


Last 


N 


j=l 
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We apply successively (9.3) and (9.4). For s=0, 


0 
x = C | P¢ Il 
is valid at the origin. 


This proves (B,) for s=0 and b —inf(d,1) (actually, it is obvious that 
dw always = 1; d—1 if and only if P is elliptic). 
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PARTIALLY ANALYTIC SPACES.* 


By Errerr BisHop. 


1. Introduction. The concept of a partially analytic space was intro- 
duced in a previous paper [1]. The present paper will refine and extend the 
theory. To this end it will be necessary to modify the definition of a partially 
analytic space which was given in [1]. Although formally quite different, 
local con- 


the modified definition contains only one essentially new feature 


nectedness is no longer required. 


DEFINITION 1. A partially analytic space consists of a separable locally 
compact Hausdorff space K and a subalgebra & of C(K), where C(K’) is the 
algebra of all continuous complex-valued functions on K. The algebra 
contains the constant functions and is closed in the topology of uniform 
convergence on compact subsets of K. % will be called the set of analytic 
functions on K. 

In the previous paper certain structure manifolds were also postulated 
in the definition of a partially analytic space. The structure manifolds are 


now permitted to occur willy-nilly. 


DEFINITION 2. A structure manifold M of a partially analytic space K 
is a subset of K which has the structure of a complex analytic manifold such 
that (a) the manifold topology and the subset-of-K topology for M are the 
same, and (b) all functions in 9 are analytic on JV. 


By manifold we mean connected separable complex analytic manifold. 

Our first task is to show that the results of [1] remain valid for the 
more generally defined partially analytic spaces considered in this paper. 
This can be done by simple modifications of the proofs given in [1]. Proofs 
are only sketched. After this has been done we study a special kind of partially 
analytic space K, called an embryonic space, which is shown to have the 
property that the image of K under a proper analytic map of K into a manifold 
Jf is an analytic subset of M. This property of embryonic spaces is then 
applied to obtain simple proofs of two known results, (i) and (ii) below. 


(i). Let A be an analytic set of dimension & in a manifold M. Let B 
le an analytic set in the manifold M—A whose dimension at each point of 


* Received May 11, 1961. 
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M—A is at least k +1. We show that B is an embryonic space. Since the 
identity map of B into M is proper, we obtain as a corollary of the proper 
napping theorem for embryonic spaces the Remmert-Stein theorem that B is 
an analytic subset of M@. For this theorem see [9]. 


(ii). Let K be a holomorphically convex analytic space and % a closed 
algebra of analytic functions on K with respect to which K is holomorphically 
convex. It is easy to see that the image K, of K under the natural map of 
K into C™ (C denotes the complex numbers) is a separable locally compact 
Hausdorff space. The algebra % can be considered as an algebra %, of 
functions on Ky, and together %, and K, constitute a partially analytic space. 
We show that this space is embryonic. From this fact we obtain Remmert’s 
proper mapping theorem [8], which states that the image of an analytic space 
under a proper analytic map is an analytic subset of the image space. 

If p is a point in a partially analytic space K, an n-tuple f,,- - -,f, of 
functions in 9% will be said to be coordinates at the point p if there exists 
a neighborhood U of p which f= (f:,- - -,fn) maps properly into an open 
set V CC” such that for each A in & there exists g analytic on V with 
h/U =gof. Then K will be said to have rank at most n at p. We show 
that an embryonic space admits a set of coordinate functions at each point. 
It follows that an embryonic space can be canonically given the structure of 
an analytic space in the sense of Serre. Using this fact and the map described 
above of a holomorphically convex analytic space K onto a canonically asso- 
ciated embryonic space K we show that if 9% is any closed algebra of analytic 
functions on K with respect to which K is holomorphically convex then there 
exists a canonical map ¢ of K onto an analytic space K, such that if %, 
denotes the set of functions f on K, such that fog€ Mf and if Xx, denotes the 
set of analytic functions on K, then %,—Wx,. This theorem extends work of 
Stein [10], and has also been proved by Cartan [4], using other methods. 

The theory of the imbedding of analytic spaces has been studied by 
Remmert, Narasimhan [7], and the author [1]. In this paper we study 
(Theorem 11) the problem of finding a proper analytic map of a_holo- 
morphically convex embryonic space K into complex Euclidean space C’ of 
sufficiently high dimension N which separates certain set of points of A and 
which locally generates all analytic functions at certain points of K. The 
imbedding theorems of [1] and [7] then follow as corollaries. The methods 
used for these constructions are basically those of [1] but also have something 
in common with the methods of [7]. 


The author is indebted to Hugo Rossi for many discussions about the 


| 
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topics of this paper. Rossi has proved by different methods some results 
which are closely related to some of the results given below, in particular to 
Theorem 6 and to Corollary 2 of Theorem 8. 


2. Partially analytic spaces. Since a new definition has been given 
of partially analytic spaces it becomes necessary to redefine analytic polyhedra. 


DEFINITION 3. A subset LZ of a partially analytic space K is called a 
frame if it is contained in the union of some countable family of structure 
manifolds of K. The dimension of L is the least integer d (possibly d =o ) 
such that Z is contained in the union of a countable set of structure manifolds 
on each of which & has dimension at most d. Let f:,- - -,f, functions in W 
and let Z be a frame. Write 


Po= {p: L,| fi(p)| <1, 1 StS}. 


Let P be a subset of P, which is both relatively open and relatively closed 
in Py. If the closure P of P is a compact subset of ZL then P is called an 
analytic polyhedron with frame L defined by the functions f,,- - -,f,. The 
dimension of P is the dimension of P when P itself is considered as a frame 
of K. P is called reduced if n= dim P, otherwise unreduced. An unreduced 
analytic polyhedron P is called prepared if it can be covered by a countable 
family {1/,} of structure manifolds such that for each 2 


(a) The dimension of 2% on VW, does not exceed dim P, 


(b) For each @ the algebra 9{ has dimension 0 on Ma over the set 


Bdry P denotes the boundary of P when considered as a subset of its frame L. 


The proof of the following theorem is the same as the proof of Theorem 
3 of [1], except for one modification which will be discussed below. 


THEOREM 1. Let {L;} bea finite family of frames in the partially analytic 
space K. Let the functions f,,- --,fn in MU define for each 7 an analytic 
polyhedron P; with frame L;. Let y=y(j) Sn be the dimensions of P;. 
Let 8S; be a compact subset of P;. Then there exist functions fy’,- fn’ 
in U, which can be taken to be polynomial functions of the functions f,,- - +. fns 
such that for each j the functions f,’,: - -,f,/ define an analytic polyhedron 
Pi with frame L; such that 8S; CP/ CP; If |fi| <1 for 1Sisn at all 
points of a compact set T C K then the functions f/ may be taken to have 


the same property. 
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Remarks on the Proof of Theorem 1. We need only consider the one place 
at which the proof of Theorem 3 of [1] needs to be modified. The modifica- 
tion is necessary because we have dropped the local-connectedness condition 
when defining the frame Z of an analytic polyhedron P. Because of the 
absence of local connectedness for Z, a union of components of P, need no 
longer be an analytic polyhedron. This means that the proof of Theorem 2 
of [1] must be modified. Theorem 2 of [1] states, in part, that if a prepared 
analytic polyhedron P is defined by functions f,,° - -,fn, if S C P is compact, 
and if 7 >1 has the property that 


S CPN {p: |filp)| 1SiSn), 
then for NV supfficiently large the functions 
(rfi(p) — (rfi(p))* 
define an analytic polyhedron Y with SC QC P. To see this write 
Qo= {p: pEL, | (rfi(p) <1, 


In [1] it was sufficient to take Q to be the union of those components of (, 
which intersect S, but this need not give an analytic polyhedron if L is not 
locally connected. Thus a different construction of Q is needed. Write 
Q,=Q@).NP. As in [1] it follows that if NV is sufficiently large then S and 
(bdry P) MQ, are disjoint closed subsets of the compact Hausdorff space Q, 
which are simultaneously intersected by no component of Q;. It follows 
(see [6], p. 158) that there exists an open and closed subset Q. of Q, which 
contains S and is disjoint from bdry P. Thus Q—Q.MQ, is an open and 
closed subset of Qo P which is disjoint from bdry of P. It follows that 
SCQCP and that Q is open and closed in Q. Thus Q is the desired 
analytic polyhedron. Thus Theorem 2 of [1] can be established even when L 
is not locally connected. The rest of the proof of Theorem 3 of [1] is the 
same. Theorem 3 of [1] is just Theorem 1 above, except for the remark that 
the f; can be taken to be polynomials in the f;. To see this note that the 
theory holds if 9 is replaced by the closed subalgebra of 9 generated by the 
functions f,,- - -,fn. This implies that the functions f/ can be taken to be 
uniform limits on compact sets of polynomials in the functions f;, from 
which it easily follows that they can be taken to be polynomials in the fi. 
Finally, it is clear from the proof of Theorem 3 of [1] that the f/ can be 
chosen to have absolute values less than 1 at all points of T. 


The following definitions and theorem correspond to Definitions 6 and 7 
and Theorem 4 of [1]. The proof is the same. 
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DEFINITION 4. A partially analytic space K is called weakly holomor- 
phically convex if K itself is a frame and if for each compact set S C K each 


component of the set 
S= {pe K: all f in %} 
is compact. 
DEFINITION 5. A continuous mapping ¢ of a topological space 7, into a 


topological space T, is almost proper if each component of ¢*(S) is compact 
for each compact set S C T>. 


THEOREM 2. Let {L;} be a finite family of closed frames of a weakly 
holomorphically convex partially analytic space K, with Ly=K. Let the 
dimension y= y(j) of L; be finite for each j. Let n=y(1) be the dimension 
of K. Then the set of elements f= (fi,---+,fn) of the n-fold Cartesian 
product %" of XM such that for each j the mapping 


fi: p> fy(P)) 
of L; into CY is almost proper is dense in YX". 


3. Special frames. Definition 6 which follows replaces Definition 8 
of [1]. 


DEFINITION 6. A continuous proper map f from a subset S of a partially 
analytic space K to an n-dimensional complex analytic manifold M is called 
special of dimension n if all of the level sets of f ou S are countable and if 
there exists a countable family {Ma} of n-dimensional structure manifolds 
of K which are subsets of S such that 


(a) fis analytic on each My and non-singular on Ve at every point of Wa, 
(8B) Each Va is open and U M, is dense in 8S, 


(y) The set H =f(S—U Ma) is a countable union of locally analytic 
subsets of 7, each of dimension at most n —1. 


It is worth noting that condition (#) may be replaced by the weaker 


condition 
(a’) fis analytic on each Ma and non-singular on WM, at some point of Ma. 


To see this equivalence it is sufficient to note that if {Wa} satisfies (#’), (8), 


| 
_ 


674 ERRETT BISHOP. 


and (y) and if Aq denotes the singular set of f on Ma then the family {J/,’}, 
where Ma’ = Ma— Aa, satisfies (a), (8), and (y). The only condition that 
is not obvious is (y), and (y) follows from the inclusion 


f(S—U Me’) C U Uf(S—U Ma) 


and the fact that f(Aa) is a countable union of locally analytic sets of dimen- 
sions at most n 1. 

Similarly we see that if the mapping f admits an extension to K such that 
S—U M, is contained in the union of a countable family of structure mani- 
folds of K of dimensions at most »—1 on each of which f is analytic then f 
satisfies condition (y). 

Before stating the next theorem we recall from [1] that if Y is a 
topological space then ,X denotes the k-fold unordered product of , con- 
sisting of all unordered k-tuples of elements of XY with the natural topology. 


THEOREM 3. Let f be a special map from a subset S of a partially analytic 
space K into a complex analytic manifold M, and let {Ma} be a countable 
family of n-dimensional structure manifolds satisfiyng («), (8), and (y). Let 
G=UMe. Then H=f(S—G) is closed and nowhere dense in M, and 
M—H is connected. There exists an integer X=1, called the multiplicity 
of the map f, such that for each point z in M—H there exist exactly x 
distinct points p in G with f(p) =z. The map from M—H into )8 
which takes z into the unordered d-tuple of such points can be uniquely 
extended to a continuous map w of M into }S8. 


Proof. The proof is the same as the proof of Theorem 5 of [1], except 
where the proof of said theorem is incomplete. In the penultimate paragraph 
of the proof of Theorem 5 of [1] it is shown that » can be continuously 
extended to (H”—H) U {z} for each z in H. To do this it is assumed 
that there are only a finite number of points p in P with f(p) 2%. This 
fact is not evident without further consideration. Assume therefore that the 
set T of points in P mapping into 2 is infinite. Since T is a countable com- 
pact set in P, we see by the Baire category theorem that T has an infinite 
number of isolated points. Choose points pi,° - *,p,41 which are isolated in 
r, and for 1 [i=A+1 let U; be a neighborhood of p; with U; NT = p; such 
that the sets U; are disjoint. As in the proof of Theorem 5 of [1] we see 
that there exists a neighborhood V of z) such that f*(V) N bdry U; is void 
for all 7. For each z in V—#H let 8;(z), 1Si1=’A-+1, be the number of 
points in w(z) NU; Since V—H is connected we see that 8; is constant on 
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V—H. Also 80 because if p is any point in P—f*(H) sufficiently near 
to p; then f(p) =z€ U; so that 8;(z) A0. Thus 


Az 
a contradiction. Thus T is finite, as was to be proved. 


Corotiary. If f is a special map from a subset S of a partially analytic 
space K into a complex analytic manifold M then the family {Ma} of 
Definition 6 can be taken to be a finite family of at most X manifolds, such 
that f(Ma) C M—H for all a. 


Proof. Let A be the set of points p in M such that w(p) consists of A 
distinct points, so that A is open and dense in 1/. Because the complement 
of A has topological dimension at most 2n— 2, A is connected. Now f-*(A) 
is a A-sheeted covering space of A. Since A is connected, the set {Ma’} of 
components of f-*(A) is finite. It is then clear that each M,’ has the structure 
of a complex analytic manifold and that {/,’} satisfies conditions («), (8), 
and (y) of Definition 6. 


Corresponding to Lemma 5 of [1] we have the following theorem and 


~ 
-~ 


corollary 


THEOREM 4. Under the assumptions of Theorem 3, let h be a continuous 
function on M which 1s analytic on M—H. Then h is analytic on M. 


Proof. The proof is essentially contained in the proof of Lemma 5 of [1]. 
H. Rossi suggests the following simple proof of Theorem 4 by induction on 
the dimension » of M. If n=—1 then H is a countable closed subset of M, 
so that the set H, of points in M where f is not analytic is countable and 
closed. Thus H, is void since otherwise H, would contain isolated points, 
contradicting the fact that a continuous function does not have isolated points 
of non-analyticity. This proves the theorem for n—1. Consider n>1 
and assume the theorem true for n—1. We may assume that M is 
En" = + 2n): Thus the intersection of H with 
all but a countable set of the varieties 


is a countable union of locally analytic subsets of L,, each of dimension at 
most n—2. By the induction hypothesis, f is analytic on all such L,. Thus 
f is analytic on WV. 


CoroLtiary. Under the assumptions of Theorem 3, let g be a continuous 
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function on S which is analytic on each Mg and let s be a symmetric poly 
nomial in variables z,,:°°,2,. For each z in M let p,,: - +, py denote the 
points of w(z), counted according to multiplicity. Then the function h on M 
defined by 

h(z) =s(g(p:)," 
is analytic on M. 


Proof. By Theorem 4, it is sufficient to show that h is analytic at each 
point z, of M—H. Let p,°,- - -,p)° be the points of w(z)). Thus 2 has a 
neighborhood U such that there exist analytic homeomorphisms 4,,° - -,¢) 
of z onto neighborhoods of p,°,- -,p,° respectively with ¢$;(z) € o(z) for 
allzin U. Thus for all z in U we have 


so that h is analytic at zo, as was to be proved. 
We now study conditions under which the image of a partially analytic 
space under proper maps will be an analytic subset of the image space. 


Lemma 1. Let f=(fi,---,fn) be a special map of a subset S of a 
partially analytic space K into an n-dimensional manifold M. Let g be a 
continuous function on 8 with values in C* which is analytic on each Mg, 
with {Ma} as in Definition 6. Let h be the map (f,g) of S into the manifold 
M=—M x Ce. Then h(8) ts an analytic subset of M. 


Proof. Consider a point r in M—h(S), so that r—(q,t,,- 
where g€ M and -,¢, are complex numbers. Let be the 
points in S which map into q under f, i.e., the points of w(q) counted 
according to multiplicity. Thus for each 1, 171A, there exists j =j(1). 
1=jSk, with 9;(qi) ~t;. There thus exists a polynomial A; in one variable 
such that A;(t;) and Aj;(g;(qi)) =0. Define the analytic function A 
on C* by 


A(m,* 
i=1 


Thus =1 and A(g(qi)) =0 for 1 It follows that 
A= Aq is a continuous function on S which is analytic on each Mg, with 
A(q@) =0 for If for each p in M we let p,,- - denote the 
points of w(p) then by the corollary of Theorem 4 we seee that each of the 
elementary symmetric functions of the quantities A(p,),-*-*,A(p,) is an 
analytic function of the point pin M. Thus the function F on M defined by 


r 
2) =I +, 2%) —A(pi)) 


— 
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is analytic on M. Clearly 
F(r) = F(q,t,° te) (1—A(q:)) 1. 


Also if (p,2:,°* is in h(S) then there exists 4, such 
that Hence F(p,2:,- has the factor 
F vanishes on h(S) and F(r) 1. Therefore h(S) is an analytic set, as 
was to be proved. 


DEFINITION 7%. An n-dimensional frame L in a partially analytic space 
K is called special if there exists a countable family {Ma} of n-dimensional 
structure manifolds of K such that 


(a) Ma for all a, 

(b) Each M, is an open set in L, 

(c) The dimension of the frame L —U Mg is at most n—1, 

(d) U Mg is dense in L, 

(e) There exists a set V D LZ such that the level sets of 2% on WN are all 
countable and such that N is a countable union of structure 
manifolds of K. 


The family {Ma} is called a structural family for the frame L. 

It is clear that if Z is a special frame of dimension n in a partially 
analytic space K then a proper map f = (f:,- -,fn) with fie 1S 
from ZL into an open subset of C” whose level sets on L are all countable is 
special whenever ZL has a structural family {Ma} such that f is non-singular 
at some point of each Ma. 

LEMMA 2. Let with gE A, be a proper 
map of an n-dimensional special frame L in a partially analytic space K into 
an open subset U of C*. Then g*(po) NL ts finite for all po in U. 

Proof. Consider a point po in U. Since g is proper, there exists an 
analytic polyhedron Q with frame L defined by functions which are linear 


combinations of the functions - -,g; such that g*(po) CQ. Since the 
dimension of % on L is n, there exist functions f,,- - -,fn in & defining an 


analytic polyhedron P with g-(p.) C P. By changing the functions f,,- - -, fn 
slightly we may assume, by Theorem 1 of [1], that f,,- - -,fn have countable 
level sets on V. Thus f is a special map of P into E”. If we let h be the 
map (f,g) of P into E" & U, we see by Lemma 1 that h(P) is an analytic 
subset of XU. Thus 


T=h(P) X {po}) =f(9* (Po) ) X {po} 
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is a compact analytic subset of H" x U C C™*, Therefore T is finite. Thus 
f(g" (po) ) is finite. Since f is at most A to one, g-*(po) is finite. This proves 


the lemma. 


THEOREM 5. If g=(91,° 8 @ proper map of an n-dimensional 
special frame L in a partially analytic space K into an open set U CC, 
where g:€ UX for all 1, then g(L) is an analytic subset of U. 

Proof. Since g is proper, g(L) is closed in U. Thus we need only check 
that g(Z) is analytic at each point p, in g(L). Let Mf, be the closure in the 
topology of uniform convergence on compact subsets of K of the set of all 
polynomials in the functions g,,- - -,g;, so that (K,%,) is a partially analytic 
space. Let V be a small neighborhood of p, with V C U. Since the map g 
is proper there exists an analytic polyhedron Q in (K,%,) with frame L with 
ILNg*(V) CQ. By Theorem 1 there exists an analytic polyhedron P defined 
by functions f,,---,fn in with LNgi(V) CP CQ. Thus f can be 


changed slightly such that each f; is a polynomial function of 9;,: - -, 9; 
Thus f is a proper map from an analytic polyhedron P D LM g*(V) into EF”, 
such that there exist polynomial functions A,,- - -,A, on C* with 


for all p in K. Since P is an open subset of the special frame J it is itself 
a special frame, so that by Lemma 2 f is a special map of P into FE”. By 
Lemma 1 we see that (f,g)(P) =A is an analytic set in BE" U, which 
clearly consists of all (A(p),p) for p€g(P), where A= (A,,:- -,A,). 
Let y:,° ° °,y; be analytic functions defined on some neighborhood Q in 
U of the point (A(po), po) whose set of common zeros is AN Q. 
Choose a neighborhood W of p,. with W CV such that (A(p),p)€Q for 
all p in W. Let B be the set of zeros on W of the analytic functions 
yi(A(p),p),° ys(A(p),p). If pe B then (A(p),p) € A so that g(P) 
Cg(L). If then (A(p),p)€ANQ so that 
yi(A(p), p) =" *=y;(A(p), p) =0 and therefore pe B. Thus g(L) NW 
= B so g(ZL) is analytic at the point jo, as was to be proved. 

DreFINITION 8. A frame J in a partially analytic space K which is the 
union of a countable family of closed special frames only finitely many of 
which intersect any compact subset of LZ is called an embryonic frame. If K 
itself is an embryonic frame then K is called an embryonic space. 


For embryonic frames we have the following immediate corollary of 


Theorem 5d. 


Corotiary. Jf L is an embryonic frame and in define a 


X 
y, 
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proper map of L into an open set U wn C* then f(L) is an analytic subset 
of U. 
As a corollary of this corollary, we obtain the following theorem. 
THEOREM OF REMMERT-STEIN. Let A be a k-dimensional analytic subset 
of an open set U CC". Let B be an analytic set in U—A of dimension at 
least k-+1 at each of its points. Then BQ U is an analytic subset of U. 


Proof. Let K—AUB and let % be the closure on K in the topology 
of uniform convergence on compact sets of all functions analytic in U. Thus 
(K,%) is a partially analytic space. Clearly L=BNOU is an embryonic 
frame in K. Let f be the inclusion map of ZL in U. By the above corollary, 
B=f(Z) is an analytic subset of U, as was to be proved. 


4. Contractions of analytic spaces. The term analytic space will be 
used in the sense of Serre (see [5], p. 280). 


DEFINITION 9. Let K be an analytic space and 2% be a closed algebra of 
functions everywhere analytic on K. Assume that K is holomorphically convex 
in the sense that the set § of Definition 4 is compact for all compact S C K. 
Let Ky be the image of K under the natural map ¢ of K into C™%. Since K 
is holomorphically convex, the map ¢ is proper and K, is closed in O%, so 
that K, is locally compact. Let %, be the subalgebra of C(K,) consisting 
of all f in C(K,) such that foge WM. It is clear that (Ko, %.) is a holo- 
morphically convex partially analytic space which we call the contraction of 
(K,%). 

Of course Definition 9 could be made for an arbitrary partially analytic 
space K, but we shall only apply the construction to analytic spaces. The 
object of this section will be to show that (Ko, %,) is actually an embryonic 
space. 

LemMa 3. With the notation of Definition 9, let M be a structure 
manifold of K on which XM has dimension n. Then $(M) is a frame in K, 


of dimension at most n. 


Proof. The proof is by induction on the dimension d of WM. If d=0 
the lemma is clearly true, so assume that the lemma is proved for all d smaller 
than the actual dimension d of M. Choose f,,---,f, in & to have rank n 
at some point of M/, and let A be the analytic subset of M at which these 
functions have rank less than n. Thus A is a countable union of structure 
manifolds of K each of dimension less than d on each of which % has rank 
at most n. By the induction hypothesis ¢(A) is a frame of dimension at 
most n. On the other hand, 1—A can be covered by a countable family 


j 
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{Ma} manifolds each of which is mapped by f1,° - -, fx onto a connected open 
set L, in C” and on each of which f,,- - -,fn and & have the same level sets. 
Thus Le is naturally homeomorphic to ¢(.V/q) ; the (Ma) are thus a countable 
family of structure manifolds of K, on each of which %{, has dimension n. 
Thus ¢(./ —A) is also a frame of dimension at most n. This completes the 


proof. 


Lemma 4. Let S be an analytic space whose set R of regular points is 
connected and let X be an algebra of analytic functions on S of dimension at 
most n at each point of R. Let M be any structure manifold of 8S. Then 
has dimension at most n on M. 


Proof. If d is the dimension of S then every level set of % on R has 
dimension at least d—n at each point. Therefore, by a known result ([3], 
Exp. XIV), every level set of 9 on S has dimension at least d—vn at each 
point. Assume that the dimension of % on M is larger than n. Thus there 
exists f= (f1,° in having rank n+1 at some point of M, 
so that f(M/) contains an open set U of C"**. Now S can be covered by a 
countable family {Ma} of structure manifolds such that for each « either f 
has rank n + 1 at every point of Ma or at no point of Ma. Let {Ma’} be the 
subfamily of {1/.} consisting of those Ma on which f has rank less than n + 1. 
If p is any point in U then f*(p) contains some level set of % on S and 
therefore has dimension at least d—n. By the theorem on the dimension of 
a countable union of closed sets, f-?(p) M Ma has dimension at least d—n for 
some a. Since the dimension of Ma is at most d it follows that f does not 
have rank n + 1 at all points of Ma, so that Ma is some My’. Thus U is covered 
by the sets f(M/.’). Since f has rank less than n+ 1 at all points of Md’, 
f(M.’) is a countable union of closed sets of topological dimensions at most 
2n. Thus U is a countable union of closed sets of topological dimensions at 
most 2n. This contradiction proves Lemma 4. 


THEOREM 6. With the notation of Definition 9, (Ko,%) is an em- 
bryonic space. 


Proof. The analytic space K is the union of a countable family {Ma} 
of irreducible sets, only finitely many of which intersect any compact subset 
of K. Thus {¢(Ma)} is a covering of K, by closed frames, only finitely many 
of which intersect any compact subset of Ky. Thus it is sufficient to show 
that each ¢(Ma) is an embryonic frame in Ky. It is therefore enough to show 
that #(7') is an embryonic frame in K, for each irreducible analytic subset 
T of K. We will do this by an induction on the dimension d of T. The 
case d= 0 is clear. Thus we assume that the statement has been verified for 


a 
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all J’ of dimension smaller than the dimension d of the given T. Let S be 
the set of singular points of T, so that dimS <d. Since S is the union of 
of a countable family of irreducible analytic sets of dimensions less than d, 
only finitely many of which intersect any compact set, we see by the induction 
hypothesis that ¢(S) is an embryonic frame in Ky. Thus if ¢(S) =¢(T) 
then @(7') is an embryonic frame, as was to be proved. Therefore we may 
assume that ¢(S) ~¢(T). We shall prove that the set 8; = TN $"(¢(S)) 
is an analytic subset of K. To this end, let p be any point in K, and let U 
he a neighborhood of p such that U is compact. Since K is holomorphically 
convex there exists an analytic polyhedron P with frame K, defined by func- 
tions f1,° * *,fn in %> such that ¢(U) C P. Now let g be any element of 2, 
where N is any positive integer and let 7 be the element of %, corresponding 
to g. Since ¢(S) is an embryonic frame in Ko, ¢(S) MP is an embryonic 
frame in the partially analytic space (P,%’) obtained by taking W’ to be all 
uniform limits on compact subsets of P of functions in Mf. By the corollary 
of Theorem 5 it follows that (f1,- - -,fn.g) =A maps ¢(S)MP onto an 
analytic set B, in EH" CN. Hence the subset 

UU (Ac $)*(By) =Ag= {pe U: € Ba} 
of U is analytic. Now clearly 

=f) Ay 

where the intersection is taken over all finite families g of functions in Y. 
Thus UM ¢*(¢(S8)) is an analytic subset of U. It follows that 6*(¢(S) ) 
is analytic subset of K. Thus 8S; =7M "'(¢(S)) is an analytic set. 

Now let & be the smallest of the local dimensions of all of the level sets 
of 2% on T. By a known theorem ([3], Exp. XIV), the set J of p in T such 
that the level set of 91 on 7 which passses through p has dimension k at p 
is an open set in 7. Thus J intersects T—S, so that MY has rank d—k at 
some point of 7’ — 

Now S, is the union of a countable family {1/.} of structure manifolds 
of K. Since the level sets of 9% on S, have dimension at least k at each point, 
by Lemma 4 we see that % is of dimension at most d—1—k on each Me. 
By Lemma 3 it follows that ¢(S,) is a frame in K, of dimension at most 
d—k—1. 

Now let S. be the set of all points in 7—~S, at which 2% has rank 
d—k—1 or less. Clearly S, is an analytic subset of T—S,. Since T—S8, 
is a (connected) manifold, S, is a proper analytic subset of T—S;. 

Now if 2 had rank d—k or more on S, at some regular point of 8, 
then %{ would have rank at least as great on 7 —S, at that point. Therefore 
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9% has rank at most d—k—1 at all regular points of S,. Thus the dimensions 
of 2 at every regular point of S, is at most d—k—1. By Lemma 4 it follows 
that S. can be covered by a countable family of structure manifolds on each 
of which 9% has dimension at most d—k—1. Thus (S82) is a frame in K, 
of dimension at most d—k—1. If S;—S.US, it follows that the frame 
¢(S;) in K, has dimension at most d—k—1. If 8s =TN¢"(¢(Ss)) it 
follows that H = T—S, is open and dense in T and that (Ss) =¢(Ss;). 
Since 9 has rank d—k at each point of H, the level sets of % on H will all 
be k-dimensional submanifolds of H or countable unions of such submanifolds. 
Now since the level sets of 29{ on H are closed in 7, and therefore compact, 
each such level set will actually be a finite union of compact k-dimensional 
submanifolds M of H. Clearly the set H of all such M has the structure of a 
d—k dimensional manifold, and % gives rise to an algebra 9% of analytic 
functions on H whose level sets on H are all finite. Also % has rank d—k 
at each point of H because % has rank d—k at each point of H. 

Now if p, and p, are distinct points in H with h(p:) =h(p.) for all 
h in %, we may choose (fis: fan) =f in having rank d—k at both 
p, and ps, so that there exist neighborhoods U, and U, of p, and pz respec- 
tively each of which f maps homeomorphically onto the same open set V C C@*. 
Thus there exists a unique analytic homeomorphism o of U, onto U. such 
that f and foo agree on U,. If h€ % then the set 


Ay, = {q€ Ui: h(q) =h(o(q))} 


is an analytic set in U,. Clearly {) A,—A is the set of all g in U, which 
are identified with some point in U, by all functions in 9. If dA=U, we 
write p=. This clearly gives an equivalence relation on H, and the 
equivalence classes form a set H which has a canonical structure of a d—k 
dimensional manifold in such a way that H is a covering manifold of 7. 
Also % gives rise to an algebra % of analytic functions on H. If p, Ap: 
in H and h(p1) =h(pe) for all h in %, define B, just as A, was defined. 
with H replaced by H and by %. Thus B is an analytic set in some 
neighborhood of p, and consists of all points in that neighborhood which are 
identified by 9 with some point in a fixed neighborhood of p,. Since for each 
p, the number of possible pz is finite, it follows that there exists a neighbor- 
hood U of p; such that all points in U which are identified with some other 
point of H by all functions in are a proper analytic subset of U. Thus the 
set W of all points in H which are identified with some other point in H by 
all functions in % is a proper analytic subset of H. 

If y denotes the natural map of H onto Hf it follows that y1(W) is a 
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proper analytic subset of H. As above we see that ¢(y*(W)) is a frame in 
K, of dimension at most d—k—1. With H,—H—y"(W) it follows 
that ¢(T —H,) = ¢(y1'(W) US,) is a frame in Ky of dimension at most 
d—k—1. Since H, is dense in T, is dense in ¢(T). Clearly 
is in one-one correspondence with H—W. It is easy to see that (HT) 
and H —W are actually analytically homeomorphic in a natural way. Thus 
¢(H,) is a d—k dimensional structure manifold in Ko, at each point of 
which %, has rank d—k. Since 7’ —H, is closed in T and therefore in K, 
¢(7T—H,) is closed in Ky. Since by construction ¢(H,) and ¢(7—H,) 
are disjoint, we see that ¢(H,) is open in the frame ¢(7’). It follows that 
$(7T) is a d—k dimensional special frame in Ky, as was to be proved. 
As a corollary, we obtain the proper mapping theorem of Remmert. 


THEOREM. If f ts a proper analytic mapping of an analytic space K, 
into an analytic space K, then f(K,) is an analytic set in Kz. 


Proof. Since the analytic structure of K, is defined by local imbeddings 
of K. onto EF‘ for some N, there is no loss of generality in assuming that 
K,—=E*%, Let Ky be the contraction of (K,,%), where % is the algebra of 
all analytic functions on K,. Since the coordinates of the mapping f are in 
%, we see that f gives rise to a proper analytic map f of A, into EY, whose 
coordinates are in %o. By Theorem 5 it follows that f(K,) =f(Ko) is an 
analytic subset of HY, as was to be proved. 

5. Imbeddings and related maps. 

DEFINITION 10. Let K be a partially analytic space. A set S C YL is 
strictly of the first category if it is contained in the union of a countable 
family {S;} of nowhere dense sets having the following property. For each 
7 there exists a compact set F; C K and a closed subset J; of (C(Fi))* such 
that S; consists of all functions in 9% whose restriction to F; is in Jj. 


THeorEM 7. Let U and V each be a countable union of structure 
manifolds of dimensions at most n of a partially analytic space K. Let 
f= (fis: fn) in have countable level sets on UUV. For each p in U 
and q in V let there exist h in M with h(p) Ah(q). Then the complement 
F of the set G=G(U,V) of all g in I" for which (f,g)(U) and (f.g)(V) 
are disjoint is strictly of the first category in MX", 


Proof. It is clearly enough to show that to each p in U and q in V 
there exist neighborhoods U’ of p in U and V’ of qg in V such that the com- 
complement of G(U’,V’) is strictly of first category. This is clearly true 
unless f(y) —f(q). Thus we may assume that U and V are mapped 
by f into the same open set [T in 0”. Let U=UU;, V=UV;, where 


| 
| 
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{U;,} and {V;} are countable families of compact sets. Let 7), consist of 
all g in &”*? such that (f,g)(U:.) and (f,g)(Vx) are disjoint. It is clearly 
sufficient to prove that for each k the complement S;, of 7), is strictly of first 
category. If we let J; be the closure of S;, in (C(U;U V;.))"*? it is sufficient 
to show that 

(a) is dense in 
and 

(b) For each g in 7; the restriction of g to U;, U V;, is not in J,. 

We first prove (a). To this end, it is sufficient to show that the set 
T of all g in %"** such that (f,g)(U) and (f,g)(V) are disjoint is dense. 


Consider therefore g = (91,° 9ns1) in W"**. Assume by induction on k 
that for each k, OS k=n+1, there exist g,’,- arbitrarily near to 
91s’ * *s 9x Yespectively such that the set of points p in U with 


for some g in V is a frame L, of dimension at most n—k, and the same for 
the corresponding frame MV, of V. This is clearly true if k=0. Thus by 
induction we may assume g,’,: already chosen, k<Cn-+1. The set of 
Jxev in & such that for each frame L of a countable family of frames of 
dimensions at most n —k covering L,;, and each frame M of a countable family 
of frames of dimensions at most n—k: covering M;, there exists a dense 
countable set of points p in L and a dense countable set of points q in V/ 
such that 
Gis (DP) € Gin’ (VOF*(F(p))) 


and 


(q) ¢ Jiu (UNF *(F(q))) 


by obvious category arguments has a complement which is a set of first 
category in M%. Thus we may find g;,,,’ arbitrarily close to g;.,. Now 
(f, Jxs1’) (M) is a countable union of locally analytic subsets of C”***!, so that 
A=LN (f, 9xs1’)"1(M) is a countable union of locally analytic subsets of L. 
Since by the choice of g;,,’ the set A has no interior in L, the frame A has 
dimension at most n—k—1. Since this is true for all Z and JV it follows 
that Z;.,, and M;.,, are frames of dimensions at most n—k—1. This com- 
pletes the induction. Letting k —n-+1 we see that (a) is true. 

To prove (b), notice that for each g in S;, the pair (f,g) identifies some 
point in U;, with some point in V;. The same property therefore holds for J,.. 
From this (b) is clear and the lemma is proved. 


CoroLuary. If L is an n-dimensional frame, if XU separates points of L, 
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and if f€ A" has countable level sets on L then the set T of all g in M"* 
such that (f,g) is one-to-one on L has a complement S which is strictly of 
first category in 

Proof. The set LX L—{(q,q): q€ L} can be covered by a countable 
family {Ux X Vi}, where U;, and V; are disjoint countable unions of frames 
of dimensions at most n. By the theorem, the complement of the set of all g 
such that (f,g)(Ux) and (f,g)(Vx) are disjoint is strictly of first category 
in %"*4, From this the corollary follows easily. 

DEFINITION 11. A point p in a frame LZ in a partially analytic space K 
is called an analytic point of L of dimension n and rank at most N if there 
exists a neighborhood U in L of p and f in 9“ mapping U homeomorphically 
onto an n-dimensional analytic subset of an open set S C C’ such that for 
each h in & there exists an analytic function w on S such that 


h(p) =w(F(p)) 
for all pin U. Then f is called a set of coordinates at p and U is the corre- 
sponding coordinate neighborhood. 
THEOREM 8. LHvery point p of an embryonic frame L is an analytic 
point of L if MX separates points of K. 


Proof. By the corollary just proved, there exist f,,- - -,fy in Y mapping 
some neighborhood V of » one-to-one and properly onto an analytic set A C EN, 
Let H consist of all continuous functions h on A such that there exists g in 
YW with g(q) =hA(f(q)) for all g in U, and let @ be the set of germs at 
z=f(p) of H. Let F be the set of germs of all functions bounded and 
analytic in some neighborhood of z on the regular points of A, so F consists 
of the germs at z of all functions analytic on the normalization of A (see 
[5]). Let R consist of all functions in some neighborhood of z on A 
which can be extended to be analytic in some 2% neighborhood of z. Thus 
kCGC F, and G and F are R-modules. By the remark at the top of p. 291 
of [5], the R-module F is finitely generated. Since Ff is noetherian, it follows 
that the sub-module G of F is also finitely generated. Thus there exist 
Ji," * *,gx in M such that for each g in % there exist h,,- - -.h, in R with 


k 
(*) G(t) = Dhi(t) 
4=1 
for all ¢ in AM V,, where V, is some neighborhood of z and g is the element 
in G corresponding to g in %. Thus if {S,} is a decreasing sequence of 
neighborhoods of z whose intersection is z, we see that for each g in % there 


exists n—=n(g) with S, C V, and (h) <n, where 


(h) =sup{|hi(t)|: 1 Sisk, te Sy} 


8 
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with h; as above. Hence % =U pp, where 


tin = {g: n(g) 
Since each p, is closed in YM, some p, contains an open subset of %, for other- 
wise 2{ would be of first category. Thus actually 8, C Vy for all g in %& for 
the right choice of Vy. Let S=S, X C*, so that fo= (f, gx) maps 
f+(S2) AU properly onto an analytic subset of S. For each g in & we have 
(*), so that 


for all g in f*(S,) NU, where 


is analytic on S. This completes the proof. 


CoroLLARY 1. Every embryonic space K for which % separates points 
admits an analytic space structure K, so that all functions in X are analytic 
on Ky. If K is holomorphically convex and if X separates points of K then 
YM consists of all functions analytic on Ko. 


Proof. The first statement is an immediate consequence of Theorem 8. 
To prove the second statement, consider any h analytic on Ky. If S is any 
compact subset of K let P be an analytic polyhedron in K containing S defined 
by f= in By Theorem 8, we may assume, by adding more 
functions if necessary, that f,,- --,f, are a coordinate set of function at 
each point of P, and by the corollary to Theorem 7 we may assume that f 
separates points of P. Thus f maps P one-to-one onto an analytic set A C Hh”. 
Let g be that function on A such that h(p) =g(f(p)) for all pin P. Since 
f is a coordinate system at each point of P and since h is analytic on Ky the 
function g on A can be locally extended at each point of A to an analytic 
function in some neighborhood of that point. As is well known, it follows 
that g can be extended analytically to #”. Thus g can be uniformly approxi- 
mated on f(S) by polynomials in the coordinate functions. It follows that /: 
can be uniformly approximated on S by polynomials in the functions f,.: °°. fy. 
Since S is any compact subset of K it follows that h € %, as was to be proved. 
The author is indebted to Hugo Rossi for this argument. 


CoroLtary 2. Every holomorphically convex analytic space K admits 


an analytic proper map onto a holomorphically convex analytic space K such 
that 


(a) The algebra % of analytic functions on K separates point of K, 


(b) M—{fod: fe 


( 
7 
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and 
(c) There are enough analytic functions in X to give a coordinate set 
of functions at each point of K. 
Proof. This corollary is an immediate consequence of Theorem 6 and 
Corollary 1 of Theorem 8. 


THroreM 9. Let p be an analytic point of dimension n and rank at 
most N in a frame L of a partially analytic space K, so that there exists 


f=(fi: fw) in WN forming a coordinate set of functions on some neighbor- 
hood U of p and mapping U onto an analytic set A in an open set M of C%. 
Let g = (91° * *59n) be an element of %" all of whose level sets on U are 


countable. Then the set T of all h=(hi,- - +,hy) in WN such that (g,h) ts 
a coordinate set of functions at all points of U has a complement S in AN 
which ws strictly of first category. 

Proof. We may assume that U is a compact subset of L. Let {W;} be a 
sequence of compact subsets of U with UW, =U. Let Gn be analytic 
functions on V with gi(p) = Gi(f(p)) for all pin U and1l1Sisn. For each 
i: let T;, consist of all h in 9(% such that there exist analytic functions ee 
on M for which hi(p) =/i(f(p)) for all p in U and 1SiS<N with the 
property that (7,4) has rank WN at each point of f(W;). Clearly TDN T;. 
Thus it is enouga to show that for each & the complement S;, of 7). in YM is 
strictly of first category. If we let J; be the closure of 8; in (C(U))*% it is 
sufficient to show that for each k& 

(a) is dense in 
and 


(b) For each h in 7; the restriction of h to U does not belong to J;. 


We first prove (a). To this end consider h in 91%. By induction on j, 
075, we show that there exist functions h,’,- --,hj in & arbitrarily 
close to h1,: - -,h; respectively and h,’,- - -,h/ as above such that for each r, 
0<r<N, the set of points in A at which the rank of (g,h,’,: - -.h/) on M 
is at most 7 has dimension at most max{—1,r—j}. This is true for j —0, 
for if g had rank r or less on a set B C A of dimension r+ 1 then the level 
sets of g on the regular points of B would not be countable, contrary to the 
hypothesis on g. Thus we need only show how to find hj,.’, assuming that 
j<N and h,’,- --,hj/ have been found. For 0SrS<WN let A, be those 
points in A at which the rank of (G, hy’, , -,h/) on M is at most r. By 
the induction hypothesis either A, is void or dimA,<r—j. We consider 
K UM as a topological space in which K and M are disjoint open sets with 
their given topologies. For 1S iN let the function /; be defined on 
KUM by fi(p) =fi(p) if pe K and f,(z)=% if z= (2° -,2y) € M. 
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Similarly define the function f;,, on K UM by fiji (p) =Ajui(p) for p in K 
and hja(z) =hi(z) for z in M, for a fixed choice of hj. Let HM be the 
closed subalgebra of C(K UM) generated by the functions /,,- -.7y, 
so that K is a partially analytic space. If p is any regular point of A,—4,., 
it is easy to see that the set of all fj,’ in 9 with the property that 
+, hj, have rank r+1 on M at p form an open dense set 
in Thus there exists h;,:’ in arbitrarily close to such that for each 
O<r<N, the functions (9, hj’) have rank r+1 at a dense 
set of regular points of A,—A,,. Let hj..’ be the restriction of hj,.’ to K 
and the restriction of hj.’ to M. It follows that .hja’) can 
have rank r or less only on the union of A,, and a subset of A, of dimension 
at most max{—1,r—j—1}; this union therefore has dimension at most 
max{—1,r—j—1}. Thus h;,,’ has the desired properties. By putting 
j= N and r= N—1 we see that (9, h’,- - -, hy’) have rank N at each point 
of A, thus proving (a). 

To prove (b), consider h in T;.. Thus h can be found so that (g,h) has 
rank N of all points of f(W;.). Now if h, in Y is close enough to / on the 
set U then by a theorem of Grauert and Remmert ([5], p. 291) ho can be 
made arbitrarily close to h on some fixed neighborhood in M of f(W;,). 
It follows that if h, is sufficiently close to h on U then hy can be found such 
that (g,h,) has rank N at all points of f(W;), so that hy is not in S;. This 
clearly implies that h is not in J;, thus proving (b). 

The following lemma replaces Lemma 6 of [1]. The proof is the 
usual proof of this type of lemma. 


Lema 5. Let L be an embryonic frame, U an open subset of L, and 
S a compact subset of U such that for each p in U—S there exists f in 
with 
| f(p)| > sup{| |: € 8}. 
Let M separate points of U. Let 1€%. Let %, be the uniform closure of 
in C(S). Then to every non-zero continuous homomorphism o of %, into 
the complex numbers corresponds p, in S such that o(f) =f(po) for all f 


im Mo. 


Proof. There exists an analytic polyhedron P with frame L, defined by 
functions f = (f:,-- -,fn) in such that SC PCU. By adding extra 
functions to f if necessary, we may assume by Theorems 7, 8, and 9 that f 
is a set of coordinates at each point of P and that f maps P one to-one onto 
an analytic set 4 C #". Thus each function in % gives rise to an analytic 
function on A which can be extended to an analytic function on E”. It 
follows that if S is identified with the set f(S) C A then 9%, may be identified 
with the set of all uniform limits on f(S) of functions analytic in BE". Let 2 


4 
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be the point z = (o(f:),° - -,o(fn)) of #". To prove the lemma it will be 
sufficient to show that z€ f(S), for then po can be taken to be that point 
of S with f(p.) —%. To this end it is enough to show that for each z in 
—f(S) there exists h analytic on with | h(z)| > sup{| h(w)|: we f(S8)}. 
This is clear if z€ H”—A for then there exists h which vanishes on A but » 
not at z. If z€ A—f(S) then by the hypothesis of the lemma there exists 
gin with | g(p)| > sup{|g(q)|: S}. If we let be any function on 
with hof=—g on P, then A satisfies the requirement. 


CoroLtuary. Under the hypotheses of Lemma 5 if 8, and 8S, are disjoint 
closed sets with S=8,US, then the function equal to 1 on 8, and 0 on Sz 
is in Wo. 

The next theorem extends Theorem 8 of [1]. The proof is the same. 


THEOREM 10. Let K be a holomorphically convex analytic space such 
that 1€ and separtes point of K. Let +,fn) be an almost 
proper map of K into C", f,E MU. Let {8;,} be a sequence of compact subsets 
of K such that US;,—K and 8S; C int Si.1 for all k. Let 


Let H;, be any compact subset of f-1(Ji.) which is open in f-1(J;,) and contains 
(S;, NM ) U A Write 


[f+ (Sx) | — 


fork=1 and For k=0,1,- - - let A, be any function in and 
any positive number. Let T be the set of all g=(91,: gv) WN such 
that for all k, k==0,1,- - - 


| 9:(p) —Ax(p)| pin G. 


Then T is not strictly of first category in UN. If the sequence {e.} 1s bounded 
and if 
inf{|Ax(p)|: p€ Gi} oo as k—oo 


then (f, 91) is a proper map of K anto (int1, 


Proof. Let {D,} be a sequence of compact subsets of K and F’, a closed 
subset of (C(D;))% such that for each & the set F;, of all g in 9% whose 
restriction to D; is in F, is nowhere dense in YW. To show that T is not 
strictly of first category we must find g in T—U F,. By re-ordering and 
padding the. sequence {D;,} if necessary we may assume that D; C H;, for 
all &. We also assume that {e,} is a monotonely decreasing sequence. If 
+,2y) is an N-tuple of complex numbers define | z | = max{| |}. 
We construct by induction a sequence g°, g'.- - - of elements in 9 such that 
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(a) + Fy for k=1, 
(b) | g*(p)| <2** min{e, +, 8.1} for k2=1 and p in where 
6; is the distance in (C(D;))* of g°+---+ gi to Fi, 
and 
(c) |g%(p) 9% (p) —Ax(p)| pe G, and k=0,1,---. 
To begin the induction take g°; *,g°w arbitrary. Assume 
that g°,- - -,g** have been constructed to satisfy the relevant conditions. 
Let U be a neighborhood of H;,, such that f* (Jes). Thus for 
each p in U—H,,, for some i with 1 Sin we have 
| fi(p)| > & +12 sup{| fi(q)|: € Hes}. 
Since G, U Hy; we see for p in 
— (Ge U Ax) (Saar) — 
that 
| fi(p)| +1>kz>sup{| fi(q)|: GU Ai} 
for some i. It follows from Lemma 5 that there exists + in W arbitrarily 
near to 0 on H;, and to 1 on G;. Write 


and g*;=0,2 N. It is clear that if + is chosen correctly then g* satisfies 
(b) and (c). Thus by changing g* slightly we may also assume that g* 
satisfies (a). 
Define g=> g*. For each k& this series converges uniformly on H, 


k=0 
because | gi(p)| < «2-41 for p€ H; whenever j>k. Thus g€ Y, since for 
any compact S C K there exists 7 with S C Sj, so that S C H;,, if k is chosen 
so large that f(S;) C Jz. We see by (b) and (c) that for all p in G; 


| 9:(p) —Ax(p)| S| +: 9% (p) —Ax(p)| 


j=k+1 g=k+1 


Therefore g€T. Also the distance in (C(D;))% of g to F;, is at least 


sup{| 9/(p)|: pe De C Hx} > 0. 


j=k+1 
This completes the proof that [ is not strictly of first category. The rest of 
the theorem is proved as in [1]. 
We turn now to a consideration of certain imbedding properties. By 
Corollary 1 of Theorem 8 we see that every holomorphically convex embryonic 


space with a separating algebra 2 of analytic functions can be realized as an 
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analytic space which is holomorphically convex relative to the given algebra 
9, and by Theorem 6 in turn we see that the analytic space can be realized 
as a holomorphically convex embryonic space with a separating algebra % 
of analytic functions. For this reason Theorem 10 holds for arbitrary holo- 
mirpically convex analytic or embryonic spaces, and the following theorem 


also yields information for such spaces. 


THEOREM 11. Let K be a holomorphically convex analytic space such 
that 1€ M and YM separates points of K. Let K have dimension at most n 
at each point, so that by Theorem 2 there exists an almost proper map 
f= fn) € of K into C™. Let Ly,---,Ly be a finite family of 
frames in K, and U,,: + -,Ux open subsets of these respective frames such 
that there exist integers N,,- - -,N;, such that each point p of Ui, 1 Sisk, 
is an analytic point of L;, of rank at most N;. Write 


N -, Nyx}. 
Then there exists g = (g1,: nm AN such that 


(a) (f,915° separates points of K, 

(b) For each 1, 1S1Sk, the functions (f.91,- +,gx,) are coordi- 
nates at each point of Ui, 

(c) (f,91:) is @ proper map of K into C"*, 


Proof. By Lemma 2 the level sets of f on K are all countable. By 
Theorem 7 the set 7, of all g for which (a) is not true is strictly of first 
category in 9%. By Theorem 9 the set 7’, of all g for which (b) is not true 
is strictly of first category in YY. Thus 7, U T, is strictly of first category. 
By Theorem 10 there exists g in K—(T,UT.) which has property (c). 
This completes the proof. 

Now we would like to improve Theorem 10 by choosing g so that the 
mapping (f1,- - -,fy,91) is proper on certain embryonic frames L in K, 
where y=dimZ and (f1,:--,fy) is almost proper on ZL. Instead of 
showing that g can he so chosen we show in the next theorem that the map 
f can be choseu so that g, automatically satisfies the extra conditions. This 
theorem will at the same time be a considerable strengthening of Theorem 2. 


THEOREM 12. Let {Lj} be a finite family of analytic sets of finite 
dimensions {y—=y(j)} in a d-dimensional holomorphically convex analytic 
space K. Then if is separating there exists f= (f1,- -,fa) in M4 such 
that 


(a) For each j the mapping fi = (fi,: +,fy) of L; into Cv ts almost 
proper, 


(b) For each j the functions +,fa vanish on 
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Sketch of Proof. By decomposing each L; into its homogeneous-dimen- 
sional parts we may assume that each JL; is a special frame. By lumping 
together all L; of a given dimension we may assume that the L; are special 
frames L,,L.,- - -,Lq of dimensions 1,2,---,d respectively. Finally, by 
enlarging the L; appropriately we assume J, C L,C--+:CLla=—K. 

Now let S be any compact subset of K for which SS. We show by 
induction on k, 0=k<=d, that there exist hi,- - -,h, in YW each of absolute 
value less than 1 on S such that for 1=jSk the functions hj,.,- - -,h;, 
vanish on Z; and such that h,,- - -,h; define an analytic polyhedron P; in 
L; with SOL; C P;. Since the case k=0 is trivial, we may assume that 
+. have been found with the desired properties. Consider p in 
K—L,.—S8S. Thus there exists w, in vanishing on with wi(p) =1. 
Since S = there exists w. in with w.(p) —1 and | w,(q)we(q)| <1 for 
all g in S. Thus there exist finitely many functions g,,- - -,gy in 9% which 
vanish on Z;_, and have absolute values less than 1 on S such that h,,- - -, Ay. 
gy define an analytic polyhedron Q in with SOL; CQ. By the 
construction used to prove Theorem 3 of [1], we may successively reduce the 
number N of such functions g, obtaining at the last step functions h,’,-- +, hy’, 
g; in 2 having the desired properties for the functions h,,- - -,h, at the k-th 
stage. Thus we may assume that the functions h,,- - -,ha can be chosen for 
each compact set S with SS. Once this is done the proof of Theorem 4 of 


[1] can be repeated word for word to construct the desired functions f,,-- -. fy. 
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ARITHMETICAL NOTES, V. A DIVISIBILITY PROPERTY 
OF THE DIVISOR FUNCTION.* 


By Eckrorp COHEN. 


1. Introduction. Let & and n denote positive integers, & fixed, and let 
7(n) denote the number of (positive) divisors of n. The set of all n such 
that &|7(n) will be denoted S; and, for real x= 1, the enumerative function 
($2) of S, will be denoted S8;,(x). Sathe [5] has investigated S;,(x) for 
odd k > 1, obtaining the estimate, 


(4.3) av as 


where a; is a positive constant defined by (3.6). Sathe’s proof depends upon 
a result [5,(3)]| of S. Selberg and Pillai which is substantially equivalent 
to the Prime Number Theorem. It is the purpose of this note to prove a 
refinement of Sathe’s estimate (Theorem 3.1). The method of the paper is 
elementary ; in particular, no appeal is made to the theory of primes. 


2. Preliminaries. The characteristic function s(n) of a set S of 
positive integers n is defined by s(n) —1 or 0 according as n€ 8 or n€ 8. 
The enumerative function S(z) of S is the summatory function of s(n), that 
is, S(v) => s(n). In this note Q will denote the set of square-free integers, 
and the characteristic and enumerative functions of Q will be denoted g(n) 
and Q(x), respectively. The symbol a,(m) will denote the characteristic 
function of the set A; of the k-th powers. The Dedekind y-function is the 
arithmetical function, 

(2.1) Hn) + 1/p), 
=n pin 
where the product is over the prime divisors of n. The number of square- 
free divisors of n will be denoted 6(n). 
We recall a previously proved lemma. 
LemMa 2.1 ([2, Lemma 5.2]). For positive integral r, 


(2.2 Q,*(2) = 9(n) 


(n,r)=1 


where B(r) = (6/n?)r/y(r). 


* Received May 24, 1961. 
693 


l- 
e 
k 
t 


694 ECKFORD COHEN. 


Remark 2.1. Clearly B(n) is bounded. 
Next we note a trivial though fundamental principle. 


Remark 2.2. An integer n is uniquely representable as the product of a 
k-free integer by a k-th power. 

Let ¢,,-(n) denote the characteristic function of the set T;,, of integers 
with a k-th power divisor whose complementary divisor is square-free and 
prime to r. Then by Remark 2.2 
(2.3) (1) -2 (d)q(8). 

(6,r)=1 
We deduce now an estimate for the enumerative function 7;,,(x7) of the 
sequence 7',,. Let €(s) denote the Riemann £-function, s > 1. 


LemMA 2.2. For positive integral r and fired k= 3 
(2.4) thr (m) = + O(a*B(r)) + O(246(r)). 


Proof. By (2.3) and Lemma 2.1, 


n=2 
= B(r)x = a, (n)n + O(a40(r) %,(n)n*) 


nSailk 
from which (2.4) results. 
Let the integer m have the canonical factorization 
(2.5) 1 t=0 if n—1, 
where p,,° * -, pe denote the distinct prime factors of n. The set B of integers 
n in (2.5) with e;>1 for all 7 is the set of “square-full” integers. Let 
B(x) represent the enumerative function of B and b(n) its characteristic 


function. 
LEMMA 2.3. 


(2.6) B(x) = b(n) = O(a). 


Proof. This estimate results from a much sharper approximation for 
B(a) proved by Bateman in [1] using an elementary approach. 


3. The main result. We introduce some further notation. The set of 
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k-free integers will be denoted Q;, while the set of k-full integers namely, 
the integers in (2.5) with ea =k (1SiSf), will be represented by By. 
In particular, Q—Q, and B=B,. If r is a positive integer =k, then F;,, 
will be used to denote the intersection, B, @Q;,, and G;,, the intersection, 
For simplicity, we write = Fr,2, and use g,(n) and 
G;,(z) to denote, respectively, the characteristic and enumerative functions 
of Gy. 

Remark 3.1. It will be observed that g;,(n) =1 if n is of the form 
(2.5) with 2S e; for all i and if &|+(n), otherwise g,(n) = 0(k = 2). 
In particular, G; C 

The following representation for s;,(”), the characteristic function of S;,, 


is basic for the main theorem. 


Lemma 3.1. If k is odd and =3, then 
(3.1) (n) = 
dd=n 
Proof. It may be seen (Remark 2.2) that an integer n has a unique 


factorization of the form, 


(3. 2) n = d,d.ds, (d,, ds) dy € F,., do € Ax, ds € Q. 


Moreover, if nm is defined by (2.5), then r(n) =(e,+1):--(e@+1), 
7(1) 1. Hence, in the factorization (3.2), since k& is odd and >1, 


n€ 8S, <=> da, € Gy. 
Thus, one obtains 
= 


dydodg=n 
(dy,d3)=1 
which by (2.3) is the same as (3.1). 
We mention two simple results that will be required. (Convention: 


vacuous products will be assumed to have the value 1). 
LemMMA 3.2. If «e<1/r(r—1), then 
(3.3) > (k=r=2) 


n=1 
né€Frr 


is convergent. 


Proof. We have 


k-1 k-1 
TT ( < TT e(i(1/(r—1) —e)). 


Thus the convergence of (3.3) is assured. 


| 
} 
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Lemma 3.3. If then 
(3.4) > 7(n)/n = O(log? x). 


For a much sharper (elementary) estimate of the sum on the left of 
(3.4), see [4, Lemma 71.3]. 


We are now in a position to prove our main result. 
THEOREM 3.1. If k is a fixed odd integer = 3, then forx=2 
(3.5) + log? x), 


where 


Proof. We first remark that the series in (3.6) is absolutely convergent, 
by the boundedness of B(m) and the case r= 2 of Lemma 3.2 with e=0. 


By Lemma 3.1, it follows that 


and hence, by Lemma 2.2 and Remark 2.1, that 


Sx = E(k) gu(n)B(n)n™ + gi (n)n-/*) 


(3.7) 
+ 


Noting again that B(n) —O(1), one obtains 


Appealing to Lemma 3.2 (r=3), it follows (for some C, > 0 depending on 
k alone) that 
n>2 m=1 n>(a/m)2 
m€ mé€ Fx, 
m=1 


Fr, 
Therefore 


(3.8) (bay + O(a). 
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Noting that G, C B, one obtains by partial summation and (2.6) 


gu(n) nk = G;,(n) — (1 + 1/n)-Vk) 


B(n)n-G4/%) + O(B(a)a-/*) 


=0( + O(a), 
from which it results that 
To complete the proof, we note the following simple properties of 
6(n): O(n) is multiplicative, 6(n?) =6(n), and 6(n) =O(r(n)) =O(n*‘) 
for alle > 0. Hence by Lemma 3. 2 with r= 3, 0 <« < 1/6, one deduces that 


(nym)=1 m€ Fk,3 
m€ Fr,3 


=0( > 0(n)/n) =0( r(n)/n) ; 


therefore by Lemma 3.3, 


(3.10) = O(log? x). 

The theorem follows on combining (3.7), (3.8), (3.9), and (3.10). 
Remark 3.2. For the case in which k is a prime, a direct evaluation of 

a, has been given by Sathe [5, Theorem 6]. For a much simpler, indirect 

evaluation of a; in this case, the reader is referred to [3]. 
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ON BOUNDARY VALUE PROBLEMS FOR LAPLACE’S EQUATION.* 


By R. T. SEELEy.* 


Introduction. The object of this paper is to formulate certain boundary 
value problems for Laplace’s equation in a way suitable for the representation 
of the boundary condition by a singular integral equation. This formulation 
is modeled to some extent on the Abel summation of Fourier series. The 
application considered is to show that, for a first order boundary operator that 
“covers” the Laplacian, the index of the problem is zero. 

Section 1 describes the regions considered, and develops the results used 
to justify the representation of the boundary conditions by singular integral 
equations. Some lemmas used in this are proved in Section 3. Section 2 
discusses the question of the index. 

The methods used here may prove applicable to wide classes of elliptic 
boundary problems. 


1. On the connection between u and du/dn for a harmonic function uw. 
Let k = 2, and G be a bounded region in k +1 dimensional Euclidean space 
E**!, with boundary T. We assume that T is connected, and is almost a C, 
manifold, of dimension k. Specifically 


i) TI isa C, manifold; the unit normal at the point x in T is denoted 

by v(x) ; 

ii) there is a number + >0 such that for every x in [ the sphere of 
radius + about x +7¥(x) intersects T only at x; 

iii) for each ¢ in 0O<|¢|<r, the points {x + tv(x); x in T} are all 
distinct and form a C, manifold T;, bounding the region G;; and 

iv) if we introduce in #* orthonormal coordinates €,,- - -,é,& with 
origin at the point x in T in such a way that the tangent plane to T at x is 
¢=0, then in a neighborhood of x, T can be represented by £=h(&,-° -,&), 
with h a C, function. 


These conditions are not independent, of course. 


It follows from these conditions that I; is of the same type as I’, with + 
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replaced by s—|¢|. Since the point x’ on I; nearest to the point x on T 
is x’ == x-+ tv(x), ¥(x%) is the normal to I; at x tv(x). 


It will be convenient to write simply v instead of ¥(x) in most contexts. 


Let do be the natural surface measure on I, and |T| = J do. Fora 
r 
function f in Z1(T), and x in let Pf (x) = cf f(y)|x—y |!*doy, where 
r 


k+1 
c= 40 ((k —1)/2) and |x—y |? => (ai— yi)”. Pf(x) is harmonic 
1 


when x is not on T; and since locally on I, P is dominated by convolution 

(in local coordinates) with a function in L*, Pf(x) is defined almost every- 

where on T. For each ¢ in 0<|¢|Sr and & in T we define P;f(x) by 

Pif(x) =Pf(x+tv). Lemma 1 proves that P;f—>P in and in 

[#(T) if fisin (1 Spo); here Pof is the restriction to T of Pf. 
An operator V, the normal derivative, can be defined on the range of Py 

by WP. f(x) =lim D;P;f(x), where D; denotes differentiation with respect to ¢. 

t>0- 


Lemma 2 shows that there is an operator K, (whose kernel is the adjoint of 
that for the double layer potential) such that D,P;f—f—+ Kf in L(T) as 
t— 0 —; the convergence is uniform if f is continuous. Thus VP) —I1-+ Ko. 
The lemma shows moreover that if f is bounded, then K,f is continuous; and 
that the kernel of K, is uniformly 0(| *—y |?“). 

The operator P, on Z*(T) has trivial null space, as the following argu- 
ment shows. Since VP»>—=I-+K , the null space of P, is contained in that 
of [+ K,. This latter null space contains only continuous functions; for 
we can choose n so that the kernel of K,” is bounded, so that if f—— Kof, 
then f = (—1)"K,"f is bounded, from which it follows by Lemma 2 that 
f+——Xf is continuous. Thus if P,f=0 for f in Z1(T), then f is con- 
tinuous. and by Lemma 1 Pf is the boundary value in the classical sense of 
the function Pf in F**. It follows from the maximum principle that Pf (x) 
=( for all x in H**, Now if ¢(x) is any function in C, on E**, with 
compact support, we have 


J. _Ao(y) dy = 0. 
It follows that f(«) =0 on I. 


We can also show that NP,f 0 only if Pof is constant. For we have 
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seen that if VP,f—0, then f is continuous; thus Lemmas 1 and 2 yield for 
u(x) =Pf(x) that 


udu/dn = lim f wau/on—lim f | Vu f |Vul?. 
r t>0- T: t>0-< G: JG 


This shows that u= Pf is constant inside G, so that Pof is constant. 

For f in L(Y) and g in L4(T) we have (NPof, Pog) = (Pof, NPog) ; 
this can be verified by approximating f and g with continuous functions and 
applying the formula 


or by using the same formula on I; and letting t—0—. Part of the same 
argument, with fg, shows that (NV Pof) 20. 


If A is the averaging operator, i.e. Af(x) =|T|- gM f(y)do,, then 


ANP,=0; for by Lemma 2 


NP.f =lim D,Pf (x + tv) dow (x’=x-+ tv), 


and the latter integral is f. du/dn for a function uw harmonic in T; and its 


interior. Now (A+N)P, has trivial null space. For if f is in L'(T) and 
(A + NV)Pof =0, then 0 = (A?+ AN)P of = APof, which gives also NP of =0. 
This last equality implies P,f is constant, and 0—AP,f shows that this 
constant is zero. Thus (A+WN)P,—IJ+K,+AP,=—I+L, where the 
kernel L(x,y) of L differs from the kernel of Ky by a continuous function 
of y; and J+ L has trivial null space. 

Now we can construct a right inverse J, for A + N, using P, as para- 
metrix. Set J) =P) + PoF, where F is an integral operator to be determined. 
(A+ N)J,=I+L+F-+ LP, so we seek F as a solution of (J + L)F =—L. 
F can be determined as an integral operator with kernel F(x,y) given by 


F(x,y) + f L (x,3)F (2, y)do,—=—L(x,y). Since J+ LZ has trivial null 
space when onlin on L1(T), this equation has, for each y in I, a unique solu- 
tion F(x,y) in I*((). The kernel /(x,y) constructed in this way has a 
singularity like that of L(x,y), and so defines an operator F on Z*(T) such 
that F + LF =—L, as desired. 

From what we have shown about P,, any function Pof in its range is the 
boundary value of the harmonic function u = Pf in G, with du/dn = NPof on 
T, both boundary values being assumed in the ZL? sense if f is in L°(T). 


é 
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Thus any function J f in the range of J) =P (I+ F) is the boundary value 
of a harmonic function u—P(I1+F)f, and on T du/on—NP,(I1+F)f 
=NJ.f=f—AJ>f. Since ANP, =0, we find Af—A?Jof or, since A* =A, 


1) Ad of = Af. 
Thus du/dn = f—Af. The function 


J (x,y) =c|x—y|t*+ cf | x |'*F (2, y) doz 
r 
with x in H** and y in T, is then the Neumann function for G. 
The results we have just obtained can be formulated as follows. 
THEOREM 1. f is in 1S p<oo, then there is a function u(x) 


harmonic in G, given by u=P(I+F)f, such that 


| Diu(x +- tv) —f(x) + Af 0, 
r 
and 
| u(x + tv) —Jof (x) 0, 
r 


as t—>0—. If f is bounded, u(x + tv) ~Jof(x) uniformly. If f ws con- 
tinuous, Dyu(x + tv) > f(x) 


Af uniformly. 


In order to state our next result, the uniqueness result corresponding to 
Theorem 1, in the best way, we need the formula 


la) J Af = Af, 


or what amounts to the same thing, that Jo>¢ = ¢ if ¢ is constant. Suppose ¢ 
constant, and let u—P(I+F)¢; then by Theorem 1 = 
This boundary value is assumed uniformly, so by classical potential theory wu 
is constant. Hence on IT, u—Jo¢ is constant. To evaluate the constant, we 
take averages and use (1) to obtain Jo6=—AJ.»6=Ad—¢, which estab- 
lishes la). 


We can now establish the following converse of Theorem 1. 

THEOREM 2. If u is harmonic in G, and f is a function in Le(L), 
(lSp<o), such that | + tv) —f(x)|?do->0 as t>0—, then 
there is a function g in L”(Y) such that f | u(x +- tv) —g(x)|?do—> 0 as 
t—>0—, and g =J,(f+ Ag). 


9 
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The theorem is easy to establish if we assume that du/dn and wu assume 
uniformly their respective boundary values f and g. For if Dyu(x + tv) > f(x) 
uniformly as 0 —, and for some g u(x + tv) — g(x) uniformly as 
then the function v(x) =u(x) —P(I+ F)f(x) satisfies dv/dn =0 on I, in 


the classical sense, so v is constant. Since f is du/dn, we have J f =0, by 
r 
(1), PUI + F)f = f Jof = J f=0. Then we can evaluate the constant 
r r r 
value of v by averaging it on I, which leads to v= Ag, u=P(I+F)f + Ag, 
and g =Jof + Ag =J.(f+ Ag). 
We can obtain the general result by applying this special case to the 


manifolds T; for <6. To this end, let x’ and y’ be on Ty, —7 St <0, and 
let L,, F;, and J; be the operators on Z'(T;) with kernels 


4 (1—k)e(x’ —y’) -¥(x") | x’ —y! | 
2) 
F,(x’,¥) ——L;(x’,¥) dow 


these are for T; what L, FY, and J, are for. Then if Au 0 in G, it follows 
that wu and du/dn-assume their boundary values on T; uniformly, so that by 


our preliminary result we have on T; 
3) u(x’) —|T; J. u(x’) doy = J. J 1(x’, y’) (0u/dn) (y’) doy. 
t t 


Now let v;(x) be such that f $(x’) doz’ + tv)v;(x)do, for every 
r 


continuous function ¢. Then v;(«) —1 uniformly in x as t->0. Since v(x) 
is the normal to T; at x + tv(x), du/dn on T; is Dyu(x + tv(x) ) ; thus relation 
(3) on T; can be written 


4) + u(x + tv) 


In order to obtain a limiting relation as {> 0—, define the operator J* on 
by the kernel 


5) J*(x,y) =I: (a + tv(x), + tv(y) 5 
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Define At on by Atf=|T;|7 f fu;do; and define uw; on by 
r 


=u(x-+tv(x)). Then according to (4), 
6) Atu, = JS * Day, for —rsSt< 0. 


Now by Lemma 3, || J'—J, || 0 as t— 0—, and the hypothesis of Theorem 
2 is that || Dw;—f || 0, where ||- || denotes the norm of an operator on, or 
a function in, It follows that in and that 
u,;—Atu; does the same. 

Thus it remains only to show that there is a function g in L?(T) such 
that u:g; it will then follow that Atu;>4dg, and g—Ag=Jp)f, or 


g=J,(f+Ag), as desired. To find g, consider first I(t) -f. u,. Then 


= Diu J f, so that /’(¢) is bounded on —r < 0 and lim/(¢) 
r r 


t>0- 


exists. Now from (6), lim|T (u,;—Atu,) =|T Jof; we have 
t0- /T 


just seen that lim f u, exists, so it follows that lim A‘u, exists. Finally, we 
r 


obtain that converges in L'(T) to g=—lim Alu; + Jof. 
t 


This concludes the proof of Theorem 2. 


2. More general boundary value problems. Theorems 1 and 2 are 
existence and uniqueness theorems respectively for the 1? formulation of the 
Neumann problem. In this section we consider more general first order 
boundary conditions, but require that du/dn exist on T in the ZL? sense, for 
some p>1. This is reasonable if the boundary operator is actually of first 
order at all points of I. 

Specifically, we look for a function wu such that 

i) Au=0O in G, 

B) ii) converges in as t->0—, for some 1< p<, 
and 


iii) Bu+tTu=h 


where B is a first order differential operator with continuous coefficients, 
defined in a neighborhood of 1; T is a bounded operator on L®(T) ; and h is 
in L?(T). In order to apply the theory of [3], we shall assume T is of class 
C;. Write Bu =adu/dn + Vu, where a is a continuous function on T, and 
Y is a continuous vector field on T. Suppose wu is any solution of (8), and 
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denote the boundary value du/dn by f. Then, by Theorem 2, there is a g in 
L*(T) such that u(a+ tv) as t>0—, and g=J,(f+Ag). Writing 
¢ for f+ Ag, we have du/On—=¢—Ad¢. Thus in the boundary condition 
Bu + Tu=h we can replace by and obtain the equation adu/dn + Vu 
+ Tu=a(¢—Ad) + + TI op = h. Now it is easy to show (as in [2], 
Section IID), that VP, is a Cy®* singular integral operator on T with symbol 
i{V,8/|$|}, where V is thought of as a cross-section of the tangent bundle 
of T, € is a vector in the cotangent bundle of I, and { , } is the bilinear form 
relating the two. Moreover the operators F, TJ), and aA are completely 
continuous. Thus H=al+VP, + is a singular 
integral operator with symbol a+ i{V,%/|$|}; and the solutions wu of the 
problem (8) are isomorphic to the solutions ¢ of the singular integral equation 


7) Ho=h 


by the isomorphism u = P(I + F)¢. 


The difference between the number of orthogonality relations that must 
be satisfied by h to guarantee a solution of (8), and the dimension of the 
space of solutions of the associated homogeneous problem: Au 0 in @ and 
Bu-+ Tu=0 on TP, is called the index of the problem (8). We obtain the 
following result. 


THEOREM 3. If a+i{V,8/|$|} does not vanish at any point on the 
cotangent bundle of T, and the dimension of T ts = 2, then the index of the 
problem is zero. 


Proof. With these assumptions, Theorem 3 of [3] asserts that the Fred- 
holm alternative holds for the equation (7), and hence for the original 
boundary value problem. Q. E. D. 


Remark 1. The hypothesis of Theorem 3 implies that a is non-vanishing 
on I, i.e. that B is nowhere tangential. If B is real, the condition that 
a-+i{V,/|$]} vanishes nowhere is equivalent to the condition that B is 
nowhere tangential 


Remark 2. The condition that a+ i{V,$/|$|} vanishes nowhere is 
equivalent to the condition that the boundary operator B covers the Laplacian, 
in the sense of Schechter (see [1]). 


3. The lemmas. In this section ||| 7'||| denotes the supremum for 
1= p= of the norm of the operator T on f(T). This norm makes the set 
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of operators 7 for which ||| 7'||| is finite a Banach algebra. The operators 
P, and P; are described early in Section 1. 


Lemma 1. ||| Py)>—P;||| 20 as t>0. If f ts bounded, Pif > Pof 
uniformly, and Pof is continuous. 


Proof. The kernel of Py) —P; is ¢|x—y|!*— c| a+ tv(x) —y|** 
= c(Q);(x,¥), where x and y are points on and | —y | is the distance from 
x to y in Let p(*,y) = (x—y)/|x+tv—y| (v—v(x)). Then a 
simple argument, based on the fact that y lies outside the sphere of radius r 
about x + rv, shows that OSp2for0S|t|S+r. It follows from this that 
|1—p| 1, and also Thus 
for the kernel of P) — P; we have cQ;(x,y), and 


| r(x, y)| =| | 1—p** | S2** | x—y|**| 1-9, 
or 
i) | Q:(*, ¥)| | x—y min (1,2 | ¢t|/|a—y]). 


Now let ¢(s) be the characteristic function of 0=s<1, and set R;(x,y) 
and S:(x,y) =Q:(x,y) Then 
S(x,y) vanishes for |*—y|<|t |4, so by (i) 


and S(x,y) defines an operator S; with ||| S;|||=0(|¢|4). Using (i) again, 
we find that on each coordinate neighborhood R;(x,y) is dominated by a 
convolution kernel (in local coordinates), and this dominating kernel comes 
from a function in Z1(E*) whose norm is 0(|¢|1/*). Thus ||| P —P;| 
=0(|t|2/*) as 0. 

P.f is continuous for f in L1(T) and ¢0. If f is in L® as well, the 
above result shows that P;f converges uniformly to Pof, and hence that Pof 


is continuous. This establishes Lemma 1. 


Remember that D; denotes differentiation with respect to ¢. 
Lemma 2. Jf f is lS p<o, then DiPif>K of +f in 


norm, as where = Ko(x,y)f(y)do,, and for 
r 

(x,y) =Dic| x + tv(x) —y |? evaluated att =0. K (x,y) is uniformly 

0(|x—y|**), and on TXT is a continuous function for r~y. If f is 

continuous, D,P;f converges uniformly to Kof +f. If f is bounded, Kof is 

continuous. 
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Proof. For 
DiPif(x) = (1— ke f(x —y doy, 


where @-f§ is the inner product of « and $8 in E**, The kernel here is 
K,(x,y) + Bi(x,y), with K,(x,y) = (1—k)e(w—y) 
and B;(x,y) = (1—k)ct|x-+tv—y|-*". Since I lies between the spheres 
of radius + with centers at x + r¥, we have for x and y on T and |x—y| <r 
that Thus K;(x,y) is 0(|x—y uniformly 
in x, y, and ¢, and an argument like that of Lemma 1 shows that if K; and 
K, are the operators with kernels K;(x,y) and Ko(x,y), then ||| K;— Ky ||| 
—0ast—0. As a consequence, if f is bounded then Kof is continuous, as 
the uniform limit of the continuous functions K;f. 

We show next that B,(x,y) satisfies (i): for each 6>0, B(x, y) 


uniformly in x and y, for | x—y| > 8, as ¢—> 0; and (ii): Bi (x,y) 1 
uniformly as {—>0— (and to —1 as t—0-++). Claim (i) is clear, so we 
turn to (ii). Introduce orthonormal coordinates (€,,- - -,&,&) in H**, with 
origin at the point x in I, so that £=—0 is tangent to T at x; and choose « so 
that 0 7, and for |x—y]| I can be represented by = h(é,,- &) 
=h(§). For simplicity of notation, we let § stand for the k-tuple such 
that (8,h($)) = (&,-: -,é,($)) are the coordinates of the point y in 
rn{|x—y|<a}. Then referring again to the two spheres about x + 7Y, 
we have the rough estimate 


Sle 
= (7 
where | Also, in {| x—y| <a} MT we have do, = v,(8) d, and 


ve($)—>1 uniformly in x, as €—0. It is not hard to show that for 
t<0, f |? + @)-@/2d§ 1, and hence that for any 8>0 
Ek 


(1— t(| |? + as t>0—. Thus for any positive 


1 <1 we can choose a positive 8< a so that for all x and | §| <8 we have 
<7? and < <7; and then we can 


choose << 0 so small that 7 << (1—k)c f Now 


let U, be the neighborhood of x in I projecting onto |é| <8. Then 


{ B,(x,y)de.— 0 uniformly in x as t—>0; and with t <0 we have for 


1 


BOUNDARY VALUE PROBLEMS. 
J B,(x,y)doy— (1—k)ct | x-+ tv—y |-*1v,(§) dé 
Ue 


that B,(x,y)doy<7**. Thus property (ii) of B,(x,y) is 
Us 


established. 

Now B;(x,y) is clearly positive for ¢<0, hence has the essential 
properties of an approximate identity. By a standard argument, B;f—f 
uniformly for any continuous function f. Then another standard argument, 
relying on the fact that continuous functions are dense in L9(T) forlSp<o, 
shows that for f in 1S p<o, in norm. This, together with 
the results for K;, establishes Lemma 2. 

It is easy to see that lim D,P;f =—f+ Kf. This with Lemma 2 is a 


t—> 0+ 


formulation of the “jump” relations for the adjoint of the double layer 
potential. 


Lemma 3. If J‘ is the operator on (LS poo) defined by the 
krenel (5), then for each p ||J'—J,)|| 20 as t-> 0. 


Proof. With reference to the kernels defined in (2), set 


and P'(x,y) =c|«’—y’ |**u(y), 


where x’ =x -+ (v(x), =y+tv(y), and 
for continuous ¢. Then 
(ay) — J y) dor 


and J*(x,y) =P*(x,y) + f We will achieve the 
r 


desired result by showing in turn that ||| P'—P,|||—0, ||| ||| 0, 
| F‘—F 0, and finally || ||p>—0. The first two cases are quite 


similar, and we discuss only P'—P,. The kernel of this operator is 
c{| x’ |?*v:(y) —|x—y |'*}, which is 0(|x—y |!) uniformly in x, 
y, and ¢, and for |x—y| 28> 0 converges uniformly to 0. Hence for any 
« > 0 we can choose § > 0 so that restricting the kernel to | x —y|<8 yields 
an operator whose norm on L?(T), for all LS po, is <«/2. We can then 
choose 4, > 0 so that, for | | < to, restricting the kernel to | x —y | = 8 yields 


rn 
4 
e 
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an operator whose norm on JL?, for all LS po, is <e¢/2. Thus for 
|t| <to, ||| P'—Po|||<«. In the same way, ||| as t>0. 

Since J+ ZL has trivial null space, and LZ is completely continuous on 
[*(T), 6+ has a unique solution ¢ for each y in L?(T), i.e. 14+ L 
is one to one and onto. For any fixed p in 1S po, it follows from the 
complete continuity of Z that the inf ||¢-+ L¢||p for || ¢ |p —1 is not zero; 
for if || dn ||p>—=1 and ¢,-+L¢,—0, we can choose a subsequence ¢’, with 
L¢’, convergent, from which it follows that for some ¢ ¢’n— ¢, with || ¢ ||p>—=1 
and ¢-+LI¢=—0. This contradicts the fact that J + ZL has trivial null space. 
Thus J-+ Z has a bounded inverse and hence for sufficiently small ¢, J +- L* 
has a bounded inverse that converges (in L? operator norm) to (1+ L)-, 
as t>0. Then (f+ converges to (I+ and 
F*) converges to J, = P,(1+F), as required in the statement 
of the lemma. 
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AFFINE RINGS OVER RANK TWO REGULAR LOCAL RINGS.* * 


By Louis J. Ratuirr, JR. 


1. Introduction. An integral domain o is an affine ring over a sub- 
domain J in case J is a Noetherian domain and 0 is finitely generated over J. 
If o is an affine ring over J and if p is a prime ideal in 0, then the quotient ring 
P =o, of o with respect to p is said to be a spot over I, hence a spot P is a 
(Noetherian) local domain. If the completion (relative to the powers of the 
product of the maximal ideals) of a (Noetherian) semi-local domain P 
contains no nonzero nilpotent elements, then P is said to be analytically 
unramified. 

A Noetherian domain J is said to satisfy the condition (SF) if each 
separably generated affine ring o over J is such that the integral closure of 0 
in its quotient field is a finite o-module. It is well known ([9], p. 267) that 
a field satisfies the condition (SF), and recently Nagata ([4], p. 419) has 
proven that a Dedekind domain satisfies this condition. Nagata’s proof is 
essentially based on the following two facts. 


1.1. Let o be an affine ring over J, and let o* be an integral domain 
which contains o and is a finite 9-module. If p* is a prime ideal in o*, then 
rank p* = rank (p* 0). 


1.2. If P is a separably generated spot over J, then the integral closure 
of P in its quotient field is a finite P-module ([4], p. 417). 


It is known ([4], p. 414) that if a local domain P is analytically un- 
ramified, then the integral closure of P in its quotient field is a finite 
P-module. Hence statement 1.2 above is a consequence of 


1.3. If P is a spot which is separably generated over J, then P is 
analytically unramified ([4], p. 416). 


In this paper, generalizing the methods of Nagata [3,4], it is proven 
that statements 1.1 and 1.3 remain valid if J is a rank two regular local 
ring which has an infinite residue field, and as a consequence it is proven 


* Received August 8, 1961. 
* This paper was written while the writer held a National Science Foundation Co- 
operative Fellowship, and constitutes the main results of his doctoral dissertation. 
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that a rank two regular local ring which has an infinite residue field satisfies 
the condition (SF). 

For the remainder of this section the letter Z will denote a rank two 
regular local ring which has an infinite residue field. 

Section 2 contains the definitions of most of the terms used in this 
paper. By employing some results of Nagata [5,6] it is proven in Section 3 
that if 9 is an affine ring over J and if q is a maximal ideal in 0, then the 
transcendence degree of o over J is equal to the rank of q minus the rank of 
qQ TJ (in symbols, trd == rankq—rank (qMJI)). Several corollaries to 
this theorem which are needed in Section 4 are then proven, and Corollary 2 
is the generalization of statement 1.1 above to the case where J is a rank two 
regular local ring. In Section 4 it is proven that statement 1.3 above remains 
true when J is a rank two regular local ring which has an infinite residue field. 
The proof is broken into three parts. Namely, let P be a spot which is 
separably generated over J. If P does not dominate J, then P is a spot 
which is separably generated over a Dedekind domain or a field, hence P is 
analytically unramified by Nagata’s result mentioned in 1.3 above. Secondly, 
if P dominates J and contains a set of elements which has a certain property 
(“property (7)”) defined below, then by means of a finite number of 
quadratic transforms of J it is shown that P is a subspace of a semi-local 
domain P* such that, if M* is a maximal ideal in P*, then P*y* is a spot 
which is separably generated over a Dedekind domain. Hence, by. certain 
well known theorems on completions of a semi-local domain ([10], pp. 277, 
283), P is analytically unramified. Finally, if P dominates J and does not 
contain a set of elements which has the property (7'), then it is necessary 
to derive some information concerning the structure of such a spot (Remark 
4.2 and Lemma 4.5). These structure theorems are generalizations of two 
theorems ([3], p. 86, and [4], p. 397) due to Nagata, and are used to show 
that there is imbedded in P a semi-local domain D such that P is a spot over 
D and D has the same quotient field as ?. Then by using a recent result due 
to Rees [8] it is seen that P is analytically unramified by proving that D is 
analytically unramified. Having proven that every spot P which is separably 
generated over J is analytically unramified, it follows that the integral closure 
of P in its quotient field is a finite P-module ([4], p. 414), and then Nagata’s 
proof that a Dedekind domain has the property (SF) carries over verbatim 
to prove that a rank two regular local ring which has an infinite residue field 
has the property (SF). 

The author expresses his gratitude to Professor H. T. Muhly for his 


encouragement and many helpful suggestions. 
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2. Definitions and notation. All rings in this paper are assumed to be 
commutative rings with a unit element 1. 


A ring o has rank r in case there exists a chain 
(2.1) CPC: HP, 


of r-+1 proper prime ideals p; in o, and there does not exist such a chain 
with more than r+ 1 prime ideals. (The symbol C is used to indicate 
proper inclusion). If p is a prime ideal in 0, then the rank of p is the rank 
of oy, and if a is an ideal in o, then the rank of a is the minimum of the 
ranks of the prime divisors of a. A chain (2.1) of prime ideals in an 
integral domain o is a maximal chain of prime ideals in o in case the rank 
of pi/Pis is one (1—1,---,7), Po= (0), and p, is a maximal ideal in o. 
The length of the chain is r. 

Let o be an integral domain. The integral closure o’ of o in its quotient 
field is the derived normal ring of o, and o is said to be normal in case 0 is 
equal to 0’. 

If a ring P has only one maximal ideal M, then this fact will often be 
displayed by the notation (P,./), and it is said that (P,M/) is a quast-local 
ring. Let (P,M) and (Q,N) be two quasi-local rings. Then P dominates 
Q(P=Q) in case PDQ and MNQ=N. If a quasi-local ring P is a 
Noetherian ring, then P is called a local ring. A Noetherian ring P which 
has only a finite number of maximal ideals is called a semi-local ring. 


Remark 2.1. Let P, Q, and RF be local rings such that P is a spot over 
() which dominates Q, and Q is a spot over R which dominates R. Then P 
is a spot over R which dominates R. 


Let (P,M) be a local ring, let y;,- - -,y; be algebraically independent 
over P, and let R=P[y,,---+,y;:]. Then the ring Ryz will be denoted by 


Yt)- 

A ring I is a ground ring in case I is a field, a Dedekind domain, or a 
rank two regular local ring which has an infinite residue field. The theorems 
which will be proven in this paper are known [38,4] if the ground ring is a 
field or a Dedekind domain. Therefore all future statements in this paper 
concerning a ground ring I are to be interpreted as statements about a rank 
two regular local ring which has infinitely many elements in its residue field. 

Let (P,1/) be a spot over a ground ring (I,m). <A ring B contained 
in P is said to be a basic ring in P in case 1) P is a spot over B, and B is a 
spot over J,2) P= B, and B= I(ynp, 3) B is a field, a Dedekind domain, or 
a ground ring, and 4) P/M is a finite algebraic extension of B/(MNM B).’ 
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Remark 2.2. Let (P,M) be a spot over a ground ring (J,m). If J' 
contains a basic ring (B,N), then P is a quotient ring with respect to a 
maximal ideal of an affine ring over B. 


Proof. By the definition of a basic ring P = Ag, where A = B[),,---, by | 
and q is a prime ideal in A. Suppose that q is properly contained in q*, where 
q* is a maximal ideal in A. Since P= B, qN B=N, and since q* q, 
q* B=N. Hence the natural homomorphism of (B/N) [b1’,: -, bn’ | onto 
(B/N) has a nonzero kernel, where and 0;* are the residue 
classes of b; modulo q and q* respectively. Therefore trd((A/q)/(B/N)) 
> trd((A/q*)/(B/N)). But B is a basic ring in P and P/M is isomorphic 
to the quotient field of A/q, so trd((A/q)/(B/N)) =trd((P/M)/(B/N)) 
=(), hence trd((A/q*)/(B/N)) <0, contradiction. Hence q is a maximal 
ideal in A, q.e.d. 

Let (R,N) be a local domain, and let (S,M) be a spot over R which 
dominates R. Then F satisfies the dimension formula in case 


(2.2 rank S + trd((S/M)/(R/N)) =rank R + trd(S/R). 


An integral domain RF satisfies the first chain condition in case every 
maximal chain of prime ideals in Ff has length equal to the rank of R. 
R satisfies the second chain condition in case every integral domain which 
contains # and is integrally dependent on F satisfies the first chain condition. 

A derivation D of an integral domain S is an additive endomorphism of 
the quotient field K of S which satisfies the conditions 1) D(xy) = «Dy + yDz, 
for all z,y¢ K, and 2) there exists an element d in S such that dDx€ 8, for 
all xe S. If Da=0 for every element a in a subdomain R of S, then D is 
said to be a derivation of S over R. 


3. The rank of an affine ring over a ground ring. As stated in 
Section 2 the statement that J is a ground ring is to mean that J is a rank 
two regular local ring which has an infinite residue field. However, the 
assumption that the residue field of a ground ring I is infinite is not needed 
in this section. 


Nagata [5,6] has proven two theorems which are stated as Remark 3.1 
for future reference. 


Remark 3.1. The second chain condition holds in a Noetherian domain 
I if and only if the first chain condition holds in the derived normal ring 
of J. Further, if I satisfies the second chain condition, and if P is a spot 
over J, then P satisfies the second chain condition and the dimension formula. 
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LemMA 3.1. Let (I,m) be a ground ring, and let p be a prime ideal 
inI. Then I, satisfies the dimension formula and the second chain condition. 


Proof. Since (0)C m is not a maximal chain of prime ideals in J, 
I satisfies the first chain condition. Since J is a normal Noetherian domain 
([10], p. 302), Z satisfies the second chain condition (Remark 3.1). Since J, 
is a spot over J, J, satisfies the dimension formula and the second chain 
condition (Remark 3.1), q.e.d. 


THEOREM 3.1. Let o=I1[%,---,2,] be an affine ring over a ground 
ring (I,m), let q be a maximal ideal in 0, and let aA I=p. Then trd(o/T) 


= rank q— rank bp. 


Remark. It is not difficult to show that there does not exist a maximal 
ideal q in o such that qQN T—(0). Hence, if gq is a maximal ideal in 0, 
then rankq—=r-+1 or r+ 2, where (o0//). Since no use is made 
of this fact in the remainder of this paper, the proof that qQ /~ (0) will 


not be given. 


Proof of Theorem 3.1. Set P=oq and M=gqP, so (P,M) is a spot 
over Iy. Since Jy satisfies the second chain condition (Lemma 3.1), every 
maximal chain of prime ideals in P has length equal to the rank of P (Remark 
3.1), hence equal to the rank of M. Since there is a one to one correspondence 
between the prime ideals in 9 which are contained in q and the prime ideals 
in P, every maximal chain of prime ideals in 9 which ends at q_ has 
length equal to the rank of q. By Remark 3.1 J, satisfies the dimension 
formula (2.2), so rank P + trd((o/q)/(I/p)) =rankI, + trd(o/7), since 
trd((P/M) /(Iy/pIy)) = trd((0/q)/(I/p)), and trd(o/I) =trd(P/Iy). Fur- 
ther, o is finitely generated over J and q is a maximal ideal in 0, hence 


trd((0/q)/(I/p)) =0. Therefore 
trd(o/Z) = rank P — rank J, = rank q — rank p, 
q.e. d. 


CoRoLLARY 3.1. With the same notation as in Theorem 3.1, if 
(0) CqiC is a maximal chain of prime ideals in 0, 
then s==rank q. 


Proof. Proved in the course of proving Theorem 3.1. 


Corotiary 3.2. Let o be an affine ring over a ground ring I, let o* 
be an integral domain which contains 0 and is a finite 0 module, and let p* 
be a prime ideal in o*. Then rank p* =rank(p*/M 0). 


| 
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Proof. Consider a maximal chain of prime ideals through p* in 0* 
ending at, say, q*. The length of this chain is equal to the rank of q* 
(Corollary 3.1). Since o* is integrally dependent on o, the contraction in 
o of the ideals in this chain yields a chain of distinct prime ideals in 0, 
and q*Mo is a maximal ideal in 0. Clearly trd(o*/J) =trd(o/J), and 
I= (q*No0)N I, so rank(q* NJ) =rank((q*N0)N JI), hence rank q* 
=rank(q*Mo) (Theorem 3.1). Since the contracted chain of prime ideals 
has length equal to the rank of q*, it has length equal to the rank of q*/ 0, 
hence it is a maximal chain of prime ideals in 0. Therefore rank(p* /M 0) 
= rank p*, q.e.d. 


CoroLiary 3.3. Let (P,M) be a spot over a ground ring I, let 
A=P[6,,: - -,0,] be a finite P-module, and let p be a prime ideal in A. 
Then rank p=rank(pfN P). 


Proof. Let Q be a maximal ideal in A which contains p, and let P = 0g, 
where q is a prime ideal in 9 =I[z] (—IJ[z,- + -,2n]). Since A is a finite 
P-module, each 6; is integrally dependent on P, hence there exist elements 
ri € 0, ri¢ q, such that the elements 7/6; are integrally dependent on o. Set 
S =o[ré], so S is a finite o-module. Clearly A > S, so 


Ag = 8 ens) 


hence Signs) A, since the elements QMS. Therefore 
Since S is a finite o-module, and since (pM S)No=(p/N P)N 0, rank(pN S) 
= rank((pM P,)NM 0) (Corollary 3.2). Since Ag and P are quotient rings of 8 
and o respectively, rank(p M S) = rank p, and rank((pN P) N o = rank(p /N P), 
q.e. d. 


CoroLuary 3.4. Let (P,M) be a spot over a ground ring I. Assume 
that P contains a basic ring (B,M), where B ts a rank two regular local ring 
which dominates I. Then rank P =trd(P/B) + 2. 


Proof. By Remark 2.2 P is a quotient ring with respect to a maximal 
ideal of an affine ring over B. Since B is a ground ring, and _ since 


rank(M B) =rank N = 2, rank P=trd(P/B) +2 (Theorem 3.1), q.e.d. 


4, Analytical unramifiedness of spots which are separably generated 
over a ground ring. The main part of this section is devoted to proving 


THEOREM 4.1 If (P,M) is a spot which is separably generated over a 
ground ring (I,m), then P is analytically unramified. 
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The proof of this theorem is attained by combining propositions 4.1, 4. 2, 
and 4.3 below. 


Proposition 4.1. Let (P,M) be a spot which is separably generated 
over (I,m). If P does not dominate I, then P is analytically unramified. 


Proof. Since P does not dominate J, the rank of MT is one or zero. 
Since a quotient ring with respect to a prime ideal of a regular local ring 
is a regular local ring ([7], p. 208), and since a rank one regular local ring 
is a local Dedekind domain, if rank MM J =1, then Ian is a local Dedekind 
domain, and if MMI (0), then Iwan is a field. Therefore P is a spot 
which is separably generated over a Dedekind domain or a field, hence P is 
analytically unramified ([4], p. 416), q.e.d. 


Henceforth the statement that P is a spot over I will mean that P ts 
a spot which is separably generated over a ground ring I and which dominates 
I. Further, the letter ¢ will consistently be used to denote the transcendence 
degree of P/M over I/m. 


Definition 4.1. Let (P,M) be a spot over (J,m). A set of ¢ elements 
4%," * *,Ye in P is said to have the property (7) in case the residues of the 
y; modulo .V are a transcendence base for P/M over I[/m, and the y; are 
algebraically dependent over J. 


Definition 4.2. Let (P,M) be a spot over (J,m). If P contains a set 
of element which has the property (7'), then P is said to be a spot of type 
one. P is a spot of type two in case P is not a spot of type one. 


Corotuary 4.1. If (P.M) is a rank one spot over (I,m), then P ts a 
spot of type one. 


Remark. There exist rank one spots which dominate 7. For example, 
let m == (a,b)I, let R=I[b/a], and let p be a rank one prime divisor of aR. 
Then R, is a rank one spot which dominates /. 


Proof of Corollary 4.1. By Remark 3.1 J satisfies the dimension formula 
(2.2), so ¢= trd((P/M)/(I/m)) =1-+ trd(P/I), hence every set of elements 
in P whose residues modulo M are a transcendence base for P/M over I/m is 
a set of element which has the property (7'), hence P is a spot of type one, 
q.e. d. 


Lemma 4.1. Let (P,M) be a spot of type one over (I,m), let 
%.° °°, Ye be a set of elements in P which has the property (T), and let 


S=I[y.,---,y:]. Then mS is a prime ideal in S, and P dominates Sys. 
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Proof. Let P = oy, where p is a prime ideal in 0 =I[z] (=I [z,---.2,]). 
Let o be the restriction to S of the natural homomorphism from P onto P/.1I. 
Since P=JI, P/M is isomorphic to (I/m) S/( MNS) is iso- 
morphic to (I/m)[y1’,- - -, ye] (the ’ indicates residues modulo WM), and 
kero =MAS2mS8S. Conversely, if f(y) is in kero, then all coefficients 
of f are in m, since the y; are algebraically independent over I/m. Thus 
kero = mS, so mS is a prime ideal, hence P= S(unsy= Syms, q.e. 


Lemma 4.2. Let (P,M) be a spot over (I,m), let P’ be the derived 
normal ring of P, let M,’,- - -,My’ be the totality of maximal ideals in P’, 
let 0; be elements in P’ such that 0:=8; (M/) (447=1,:--,h). and let 
A=P([6,,---,6,]. Then A is a semi-local domain which has exactly h 
maximal ideals M/A. Let P;=Ag,. Then the derived normal ring 
Pi of Py ts 


Remark. The fact that P’ has only a finite number of maximal ideals 
is proven in [2]. 


Proof. Set Q:x=M/ NA. Since A is integrally dependent on P, the 
derived normal ring A’ of A is equal to P’, hence each Q; is a maximal ideal 
in A, and the Q; are distinct by the choice of the 6;. Therefore there exist 
at least h maximal ideals in A. If Q is an arbitrary maximal ideal in 4, 
then there exists a maximal ideal M’ in P’ such that M’N A =Q, since P’ 
is integrally dependent on A. Since J/’ is one of the MM’, Q=M/NA=Qi. 
Thus there exist exactly 4 maximal ideals in A. Also A is Noetherian, since 
P is Noetherian, hence A is a semi-local domain. Fix i and set R=P’y,, 
N=M/R, and contains P/, since R is normal). Then 
R= (P7)y. If q is a maximal ideal in then qN A’ =qn P’ = My, 
(P/)gZ= RZ (Pi’)y, hence and Pj = (P/)y—R, q.e.d. 


Remark 4.1. With the same notation as in Lemma 4.2, if d,,- - -,d 
are elements in P;’, then S=Pi[d,,- - -,d,] is a spot over J, and rank S 
=rankP. Further, trd((S/Q)/(I/m)) =trd((Pi/Mi)/(I/m)), where @ 
and M/; are the unique maximal ideals in S and P,; respectively. 


Proof. The proof that S is a spot over J follows from Remark 2.1 and 
the fact that P,’ is a quasi-local domain, and rank § = rank P by Corollary 3.3. 
The last statement is clear, since the d; are integrally dependent on Pj. 


Lema 4.3. Let (P,M) be a spot of type one over (I,m). There 
exists a semi-local domain P* which is contained in the derived normal ring 


a 
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of P such that 1) P* contains P and is a finite P-module, and 2) if M* is 
a maximal ideal in P*, then P*y* is a spot which is separably generated over 
a Dedekind domain. 


Proof. Let P’ be the derived normal ring of P, and in the notation of 
Lemma 4.2 let A= P[6,,- - -,0,]. Then by Lemma 4.2 A is a semi-local 
domain which has h maximal ideals Q; = MN A, P,’ is a quasi-local domain, 
and by Remark 4.1 P; is a spot over J (t—1,---,h). Let y1,---+,y bea 
set of elements in P which has the property (7). Since the 6; are integrally 
dependent on P, it is clear that y,,- - -,y: are elements in P; which form a 
set which has the property (7). Further, if an element d is in the derived 
normal ring of P;, then there exists an element r€ A, r¢ Q; such that rd is 
an element in the derived normal ring of A. Hence, in view of Remark 4.1 
it is sufficient to prove the lemma in the case where the derived normal ring 
of P is a quasi-local domain (and in this case the ring P* is a local domain). 

Let LZ and F be the quotient fields of P and J respectively, let (#,, M,) 
be the valuation ring of a real discrete valuation v of LZ such that Rk, =P 
(existence of FR, is proven in [1]), let w be the restriction of v to F, and let 
(Tu, Nw) be the valuation ring of w. Then clearly T, =J, hence w is 


not trivial on F. Let Yi,---,Y; be indeterminants, and extend w to 
I[Y] (=1[¥3,: +, by defining w(g(V)) =min{w(7;) ; 7 is a coefti- 
cient of g(Y)}. Since y,,---+,y, are elements in P which form a 


set which has the property (7), there exist polynomials f(Y) in I[Y] 
such that f(y) 0 and reducing modulo M the equation f(y) =0 it 
is seen that all coefficients of f(Y) are in m. Among the polynomials 
f(Y) such that f(y) =0 let g(Y) be such that w(g(Y)) is a minimum 
(such a polynomial exists since T,=J). Let g(Y) =D riMi(V), where 
J,(¥Y) is a monomial with a coefficient which is a unit in J. Then w(g(Y)) 
>0, so it may be assumed tat each 1,€ m4, and that at least one 1; ¢ m*. 
Let m= (a,b)I. Then > and since at least one at 
least one rj; is a unit in J. Set so g(Y) = 
Further, w(¢;(Y)) =0, for some j, hence ¢;(y) 40. 

Under the unimodular transformation b,=b, a, =a—cb,, where c is 
a unit in J, the coefficient of 6,4 in g(Y) is S¢;(Y)c®&%. Since I/m is 
an infinite field, there exist values of c such that the coefficient of 0, 
(= ~ (= rjc®-) M;(Y)), is such that for at least one +> rjc®-) is a unit in J, 


4)(y) 4A~0. Thus it may be assumed that the coefficient ¢g(Y) 
of b? in g(Y) has the property that w(¢g(Y)) —0, hence og(Y) has a 
coefficient which is a unit in J. Therefore $g(y) is a unit in Sms, where 


10 
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S=I[y.,---,y:] (mS is a prime ideal by Lemma 4.1). By Lemma 4.1 
P= Sms, hence ¢g(y) is a unit in P. Hence, from 


it is seen that b/a is integrally dependent on P. 

Set Py» =P, P,=P,[b/a], and let M, be the maximal ideal in P,, 
Since P” is a quasi-local domain, P, is a local domain contained in P’ which 
is a finite P-module and which dominates P. Set J, =I, J =J,[b/a], 
won =Jweny, and Then (P,,4,) is a spot 
(which is separably generated) over (J,,m,), and P;=J,. (J; is called the 
first quadratic transform of I along w. It is known ([4], p. 404) that J, is 
a regular local ring which dominates J, and rank J; Srank/. Let J; be the 
first quadratic transform of (tk 21) along w (assuming rank > 1). 
Then J;, is called the k-th quadratic transform of I along w). If rank J,—1, 
then the theorem is proved. If rank/,—2, then y;,- - -,y: are elements 
in P, which form a set with the property (T), Ry, 2 Pi, Ty 21,, and the 
derived normal ring of P, is P’, so the above process can be repeated. Let 
gi(Y) be an element in J,[¥Y] such that g.(y) =—0 and w(gi(Y)) is a 
minimum. Then w(g,(V))>0. Since m8 C = a8y;, where p; € 
so g(Y) =af(Y), where f(Y) = Since g(y) =0, f(y) =0, hence 
w(g(Y))>w(f(V)) Zw(gi(VY))>0. Since w(g(¥)) is an integer, in a 
finite number,of steps, say k, rank Ij, = 1, where J;, is the k-th quadratic trans- 
form of J along w, and P; is a spot (which is separably generated) over /;.. 
Since J;, is a regular local ring of rank one, J, is a local Dedekind domain, 
q-e. d. 


Proposition 4.2. Let (P,M) be a spot over (I,m). If P is a spot 
of type one, then P is analytically unramified. 


Proof. Let P* be the semi-local domain constructed in Lemma 4.3, 
and let 1/* be a maximal ideal in P*. Then P*y* is a spot which is separably 
generated over a Dedekind domain, hence P*y* is analytically unramified 
({4], p. 416). Since the completion of P* is isomorphic to the direct sum 
of the completions of the P*y* ([10], p. 283), P* is analytically unramified 
Since P* is an integral domain which is a finite P-module (Lemma 4.3), the 
completion of P is a subring of the completion of P* ([10], p. 277), hence 
P is analytically unramified, q. e. d. 


CoroLiary 4.2. Let (P,M) be a rank one spot over (I,m). Then P 
is analytically unramified. 


Proof. By Corollary 4.1 P is a spot of type one, q.e.d. 
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Let (P, M) be a spot of type two over (I,m). The every set of ¢ elements 
y:;°° *>¥t in P which map modulo M onto a transcendence base for P/M 
over I/m are algebraically independent over I, hence the ring B = I(yi,- - - , yt) 
is a basic ring in P. (B is a rank two regular local ring, since the y are 
algebraically independent over I). Since P is a spot over I, P ~0g, where q 
is a prime ideal in an affine ring o over J. Since P/M is isomorphic to the 
quotient field of o/q, there exist y:,- - -,¥:€0, ¢q such that the y; map 
modulo M onto a transcendence base for P/M over I/m. 


Definition 4.3. Let P =odgq be a spot of type two over (I,m). A basic 
ring B in P is a special basic ring in P in case B=I(y;,- - -, yt), where 
41° °°, Ye are elements in 0, not in q, whose residues modulo q are a trans- 
cendence base for 9/q over I/m. 


Remark 4.2. If P is a spot of type two over J, then P contains a special 


basic ring. 


Remark 4.3. Every special basic ring in P is a rank two regular local 


ring. 


Lemma 4.4. Let (P,M) = (09, qQ0q) be a spot of type two over (I.m), 
let rank P=—r, and let (B,mB) be a special basic ring in P, where 
Then there exist r—2 elements in gq which are alge- 
braically independent over -,y¢]. 


Proof. Since P is a spot of type two, r22 (Corollary 4.1). Since 
rank P = trd(P/B) +2 (Corollary 3.4), trd(0/A) =r—2. If r—2, then 
there is nothing to prove, so it may be assumed that r > 2. Hence there exists 
an element 2, € 0 which is transcendental over A, and it may be assumed that 
2,¢q. Then the residue 2,’ of z, modulo q is algebraic over A/(qM A) (by 
the choice of the y;). Let f,’(21’) =0 be a (not necessarily monic) minimal 
equation of z,’ over A/(qN A), and let be an element in q 
which maps modulo gq onto f,’(2,’). Since z, is transcendental over A, 2, is 
transcendental over A. If r=38, then the lemma is proved, so it may be 
assumed that r > 3. Then there exists an element z.€ o such that z. is trans- 
cendental over A[2,], and it may be assumed that 2o¢ q. As above let f.(z2) 
=. be an element in q which maps modulo q onto a minimal equation 
fo’ =0 of 2.’ over A/(QN A). Then 22~2,, and since z, is 
transcendental over A[2,], v2. is transcendental over A[2,], hence 2, and 2, 
are algebraically independent over A. Hence in 7—2 steps the required 
elements are obtained, q.e. d. 
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Lemma 4.5. Let (P,M) be a spot of type two over (I,m), and let 
rankP=r (r=2). Then there exists a special basic ring B* in P and 
elements 2;,° * *,%r-2 in M which are algebraically independent over B* such 
that P is a finite separable algebraic extension of B*[z,,- - +, 2-2]. 


Remark. The proof of this lemma is a direct generalization of Nagata’s 
proof of Proposition 8 ([4], p. 397). 


Proof. Since P is separably generated over J, when the characteristic 
of J is 0 the lemma is true by Lemma 4.4, so it may be assumed that the 
characteristic of J is pA~0. Let and let y+) bea 
special basic ring in P (B exists by Remark 4.2). Let A=I[y1,-° - -.y], 
and let 2,° - *,2;2 be elements in q which are algebraically independent 
over A (Lemma 4.4). Then trd(o0/A[%,- -,2-2]) =0, hence trd(o//) 
=t+ (r—2). Let n=t+r—2, let (1—1,- -,r—2), and let 


Wroi=Yi (it=1,:--,¢). Since P is separably generated over I. let 
D,,- - -,D, be a maximal linearly independent set of derivations of o over J 
(existence is proven in ([4], p. 390)). Let w,- - +, un be elements in o such 
that Dywu;~0 if and only if ij. It may be assumed that u;—1¢€q 
(j7=1,:--,m), since otherwise the elements 1+ uja? (a#€ q) are such that 
D,(1 + uja?) AO if and only if 1+—j, and (1+ uj;a?)—1 are in q. Set 
zj—= wu; Then Dyz;0 if and only if hence 2,---,2, are a 
separating transcendence base for over J. Since u;—1€ q, 2/ = wj’?uj = w/? 
(where the ’ indicates residues modulo gq). Now w/?=0 (j—1,: -,r—2), 
and the w/? (j—=r—-1,: --,n) are a transcendence base for 0/q over I/m. 
Therefore the ring B* =1(2,,,: - -,2n) is a special basic ring in P, and the 
elements 2;,° - *,2,-2 are elements in q which are algebraically independent 


over B*, q.e.d. 


Proposition 4.3. Let (P,M) be a spot over (I,m). If P is a xpot 
of type two, then P is analytically unramified. 


Proof. Let rank P=r (r=2). Let (B,mB) be a special basic ring in 
P such that there exist elements 2,,- - -,2;-2 in M such that P is a finite 
separable algebraic extension of C* —B[2,,- - (Lemma 4.5). Set 
Q* = (mB, -,%-2)C*, and @=Q*C. Since B is a regular 
local ring of rank two (Remark 4.3), and since the 2% are algebraically 
independent over B (Lemma 4.5), (C,@Q) is a regular localy ring of rank 7, 
and P=C. Let £ be the quotient field of C, and let L be the quotient field 
of P, so L is a finite separable algebraic extension of Z. Hence, since C is 
a normal Noetherian domain, the integral closure C’ of C in L is a finite 
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(-module ([9], pp. 264-265). Therefore the integral closure D of C in P isa 
finite C-module. Since P is a spot over B, there exists elements d,,---,d,€ P 
such that P is a quotient ring with respect to a prime ideal of the ring 
C{d,,: + *,d,], and since the d; are algebraic over EH, there exist elements 
s,¢ C such that the elements s;d;€ D, hence the quotient field of D is L. 
Since P= D(unvd) = C, MN Dis a maximal ideal in D and there exist elements 
e,.° * *,@,€ LZ such that P is a quotient ring with respect to a prime ideal 
of Durap)[e:,: - -;¢]. Hence by a recent result due to Rees [8], P is 
analytically unramified if D (yzqp) is analytically unramified. Since D is a 
finite C-module, D is a semi-local domain, hence D.yqp) is analytically un- 
ramified if D is ([10], p. 283), and since C’ is a semi-local domain which is 
a finite D-module, D is analytically unramified if C’ is ({10], p. 277). Hence 
to prove that P is analytically unramified it is sufficient to prove that C’ is 
analytically unramified. 

Since L is a finite —* algebraic extension of H, let u be an element 
— bw, and set H Then H is an integral 


i=0 
domain which saiiians: C and is contained in (’, hence H is a semi-local 


domain with quotient field L. Let F' be the total quotient ring of the com- 
pletion of H. Since C’ is a finite C-module, there exists a nonzero element 
d€C such that dC’ C H. Since the completion of C is a regular local ring 
({10], p. 302), d is not a zero divisor in the completion of H ({10], p. 277 

hence F contains the completion of C’. Let K be the quotient field of the 


in C’ such 


completion of C. Then it is seen that F—>Ku'. Since F is a finite 
4=0 
dimensional algebra over K which contains the completion of C’ and is con- 
tained in the total quotient ring of the completion of C’, F’ is the total quotient 
ring of the completion of C’. It is a straight forward verification that F is 


isomorphic to and since L is separably generated over L® K con- 


E 
tains no nonzero elements ([9], p. 195). Therefore is analytically 
unramified, hence P is analytically unramified, q.e.d. 


Propositions 4.1, 4.2, and 4.3 constitute a proof of Theorem 4.1. 
It is known ([4], p. 414) that if a semi-local domain P is analytically 
unramified, then the derived normal ring of P is a finite P-module, hence 


Corotiary 4.3. If P ts a spot which is separably generated over a 
ground ring I, then the derived normal ring of P is a finite P-module. 


As mentioned in the introduction, Nagata’s proof that a Dedekind domain 
has the property (SF) is essentially based on statements 1.1 and 1.2. Since 


d 
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Corollary 3.2 and Corollary 4.3 are respectively the generalizations of these 
statements to the case where J is a rank two regular local ring which has an 
infinite residue field, Nagata’s proof carries over verbatim to prove 


THEOREM 4.2. If 0 is an affine ring which ts separably generated over 
a ground ring I, then the derived normal ring of 0 ts a finite o-module. 
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UNIFORMLY BOUNDED REPRESENTATIONS, 


Analytic Continuation of the Principal Series of Representations 


of the n X n Complex Unimodular Group. 


By R. A. Kunze? and E. M. STeEIn.? 


0. Introduction. This paper is the second of a series devoted to the 
study of uniformly bounded representations of classical groups and the 
application of these representations to analysis on these groups. 

In the previous paper [8], this was carried out for the special case of 
the 2 X 2 real unimodular group. Two related results were basic. The first 
dealt with the analytic continuation of the ‘ principal series’ of representations 
—leading to a family of uniformly bounded representations defined in a strip. 
The second result applied the analytic family of representations to obtain a 
‘Hausdorff-Young Theorem’ for the group together with its implications for 
the characterization of representations. 

Our purpose here is to carry out the analytic continuation of the prin- 
cipal series in the case of the group of n X m complex unimodular matrices. 
The problems dealing with the continuation for other semi-simple groups 
and the problems connected with the Hausdorff-Young Theorem will be con- 
sidered in future papers of this series. 

We consider the group G of n Xn complex unimodular matrices, and 
its diagonal (i.e., Cartan) sub-group C, whose entries we denote by 
C= (C1, C2," * *5€n), C€ C. Let A denote a continuous character of the sub- 
group C. Then 


c, \™ \me Cc Mn 
| | en | 
Where 83, *,S, are complex numbers and m,,ms,: -,m, are integers; 
both sequences are uniquely determined if we require that s,+s,+-:--+s, 


(We call the integer r, r= m,+ m, 
+::+:tm,, 0OSr<n, the residue of the character A.) Notice that the 
character is unitary if Re(s;) =0, -,n. 


* Received August 31, 1961. 
* Research supported in part by National Science Foundation Grant No. 12847. 
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Gelfand and Neumark have shown how to construct an irreducible unitary 
representation for each unitary character A. This leads to the ‘ principal series’ 
of representations ; these are the representations which arise in the Plancherel 
formula for the group. They are constructed as follows. 

For each such A the representation a— 7'(a,A) is realized as a multiplier 
representation on L,(V), where V is the sub-group of G consisting of matrices 
which have ones on the main diagonal and whose entries above the main 
diagonal vanish. The representation T'(a,2) is defined by 


(0. 2) T (a,r): f(v) > K (va, A)f (va) 


where K is an appropriate multiplier and the mapping v > vd may be thought 
of as a generalization of fractional linear transformations which occur when 
n= 2. 

Now the multiplier appearing in (0.2) depends on A and thus on 
81,82," * *,S,. This dependence may in fact be seen to be analytic. Thus we 
can obtain in a trivial sense an analytic continuation of (0.2). This naive 
analytic continuation is, however, not the right one for a variety of reasons. 
The simplest of these is that the resulting operators are not uniformly 
bounded, in fact not everywhere defined on L?(V) when Re(s;) ~0.* 

The natural analytic continuation may be motivated by the following 
considerations. 

We quote two known facts about the representations (0.2), (when 
ve(s;)==0). The first is that they already irreducible on a certain sub- 
group G, of G. The second fact is that T(-,A) and 7T(-, pA) are unitarily 
equivalent representations for any character A and any permutation p of the 
character (i.e., element of the Weyl group of @). 

It is convenient to group the characters A into n distinct classes— (their 
residue classes). The class to which any character belongs is determined by 
the unique integer r, OS r<n, for which m,+ m,=". 
Or put another way, the class is determined by the action of » on the center 
of G. 


We come now to what is in effect a basic result of this paper. The 


*It is possible to bypass the last objection and obtain by analytic continuation 
everywhere defined (but not uniformly bounded) representations by reformulating the 
representations (0.2) in terms of the maximal compact sub-group. See Harish-Chandra, 
“ Representations of a semi-simple Lie group I,” Trans. Amer. Math. Soc., vol. 75 (1953), 
p. 241. It should be added, however, that in this sense there are infinitely many “ ana- 
lytic continuations ” of the principal series. Only by adding certain natural considera- 
tions can one find a unique analytic continuation, which in addition, enjoys the further 
properties which are described hereafter. 
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representations 7'(-,A,) and T(-,A2) for which A, and A, belong to the same 
class are equivalent when restricted to G. This is remarkable in view of the 
fact that they are already irreducible on Gp). Thus there exists a unitary 
operator, W(A), so that if we define R(a,A) by 


(0.3) R(a,A) = W(a)T (a, 


and if we fix a€ G, then R(a,r) depends only on the class of A (and thus in 
particular is constant as a function of s;,52,- - -,8,). In view of the already 
quoted irreducibility of T(-,A) restricted to Go, the representations R(-,.) 
so determined are unique. We call these the normalized principal series. It 
is these representations that can be continued analytically in the parameters 
of X while T(a,) and W(A) separately can not. 

The discussion above insures the existence of W(A) and thus R(a, A) 
a priort, but tells us nothing about their concrete properties which are necessary 
for analytic continuation. We need first, therefore, construct W(X) in explicit 
terms. 

In order to get an idea of how to go about this, we recall, as mentioned 
above, that 7'(-,A) and T(-, pA) are unitarily equivalent; therefore R(-,A) 
and (R-, pA) are unitarily equivalent. Using the fact that R(a,’A) = R(a, pa) 
for a@€ Go, it follows (from the irreducibility on G,) that R(a,A) = R(a, pa) 
for all a€ G, and finally W-*(pA)W(A)T(a,A) =T (a, pA) W-*(A)W (pa) for 
all p in the Weyl group. 

This shows that an explicit determination of W(A) brings with it a 
similarly explicit description of the intertwining operators A(p,A) defined 
by A(p,A)T(a,A) =T (a, pA) A(p,d). 

We reverse the heuristic argument described above and begin our study 
by obtaining the intertwining operators A(p,A) in a concrete fashion. It 
turns out that this task is much simplified when we restrict ourselves to the 
n—1 permutations po,* Where p, corresponds to the transposition 
(7,7-+1) of the symmetric group on n letters. These n—1 permutations 
generate the whole Weyl group. It is then found that the operators A (p,, 2) 
can be given by singular convolution integrals whose kernels are carried on 
one-dimensional (complex) sub-groups of V (see Theorem 1, Section 4). 

A basic property of the operators A(p,A) is that although they do not 
commute in general, they satisfy certain fundamental commutativity relations 
(see Theorem 2, Section 5). These commutativity relations are a reflection 
of the relations satisfied by the generators p,, Pn-1 of the Weyl group. 
The actual analysis of the commutativity relations is rather complex but is 
simplified by reducing the problem to the case of the 3 X 3 group by a change 


‘ 
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of variables. In this sense the 3 X 3 group is already the general case, because 
here we must deal with the non-commutativity of the sub-group V (3), which 
makes the Fourier analysis on L,(V) difficult. 


Once the properties of the operators A(p,A) are obtained, we can then 


) 


define the operators W(A). These are given by products of “—) operators 


A(p,A), (see 6.5) and 5.17)). We then prove in Theorem 3 (Section 6) 
the basic properties of the normalized prinicipal series for unitary 4. 

Our next task is the analytic continuation of this series. The problem 
of analytic continuation is tractable because of the following two considera- 
tions. 1) It suffices, in effect, to continue one parameter only keeping the 
others fixed. That this allows us to obtain analytic continuation simul- 
taneously in all parameters follows from the lemmas of Section 8. 2) The 
group G is generated by the sub-group G, and a fixed element pe G. Since 
the R(a,rA), a€ Go, have trivial (i.e., constant) analytic continuations, it 
suffices to consider the analytic continuation of the single operator R(p, Ad). 


Here use is made of the lemmas of Section 7. 


The resulting uniformly bounded representations are defined when the 
parameters s;, 2, * *,S, Vary over an open ‘tube’ in the complex hyperplane 
8: +8.+:-+-++5s,—0. Their basic properties are set forth in Theorem 4 
of Section 9. 

In Section 10 we calculate the ‘trace’ of these representations which 
allows us to prove in particular the following: (i) R(-,A) is unitary if and 
only if A’ = p(A), where ’ is the complex contragredient character of A, and 
p is some element of the Weyl group. Hence R(-,A) is unitary if and only 
if it corresponds to an element of the principal or complementary series. 
(ii) If X’Ap(A), then although R(-,A) is uniformly bounded, it is not 
equivalent to a unitary representation. 


1. The principal series. In this section we recall relevant facts about 
G@ and the construction of the principal series. Detailed proofs of most of 
the results we state below may be found in the book by Gelfand and Neumark 
[3].° 

Let GL = GL(n) denote the group of all complex non-singular matrices 
of degree n. In what follows we shall need to consider the following sub- 
groups of GL. 


%In stating the results of Gelfand and Neumark we have found it convenient to 
use a notation which is different from theirs. 
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G = G(n): the unimodular sub-group. 
C =C(n): the (Cartan) sub-group of diagonal matrices c = [¢, C2,° ++, Cn] 
in G. 


U =U(n): the upper triangular unipotent matrices 


1 Un 


LO 
Uj, =0 for and uj—1. 
V = V(n): the lower triangular unipotent matrices 
0 
v= 
| 


Vj, = 0 for 7 << k and vj; —1. 


( ja 
ky lem 


For each a in GL, let 


be the determinant of the submatrix of a formed from rows 7;,° - -,j» and 
columns k,,: - -,km of a. Put 
D,(a) ") 
Daya (a) =1. 


Then in order that a may be expressed as a product 
(1.2) a=ucv, ue U, ve V, diagonal 


it is necessary and sufficient that D;(a) 40 for 1SjSn. The matrices a, 


v, and c are then uniquely determined as follows: Let 


(1.3) ice 

(1.4) 
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Then u, v, and ¢ are given by 


Dix,(@) 


1 
(1.6) (a) j>k 

(1.7) 1<j<n 


If a€ GL and all the determinants D;(a) are different from 0, we shall 
say that a is decomposable. The diagonal matrix c in (1.2) will be called 
the diagonal part of a and will be denoted by diag(a). It should be noted 
that diag(a) belongs to C, i.e., has determinant 1 if and only if a is in G. 

The elements v of V are parametrized by the real and imaginary parts, 
Liz, Ys Of their entries v;,. The measure dv obtained by taking the product 
of the differentials 
(1.8) Av jx, == dxj,dyjx, 7 >k 


is both left and right invariant; so V is unimodular. 

In the 2 & 2 case, V may be identified with the additive group of complex 
numbers, and GZ then acts on V as the group of linear fractional transforma- 
tions. In the general case GL acts on V in a similar fashion. We shall now 
describe this action. Let a be a fixed member of GL. Then for v in V, the 
determinants D;(va) are polynomial functions of the entries vj, of v. Thus, 
with the exception of a iower dimensional set of measure 0, the product va 
is decomposable ; the exceptional set of elements v depends on a. If va can 
be expressed as a product u’c’v’ with u’ € U, v’€ V, and c’ diagonal, then this 
can be done in only one way and we shall put 


(1.9) va =v’. 
In the 2 2 case, v@ is the element of V defined by 


+ 
+ 


(1.10) (va) 


When n is arbitrary, there are still some instances in which 7d@ is easily 
described. For example, if ¢ is diagonal, ve=c-1vc, and for any v,€ V, 
V0, = 

Let v€ V, a€ GL, and suppose va is decomposable. If 6 is any member 
of GL, then vab is decomposable if and only if (v@)b is; when this is the 
case it is then true that 


(1.11) v(ab) = (va) 
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When n=2, the mapping v—> vd induces a transformation of the real 
parameters corresponding to v2: whose Jacobian is equal to 


| det a | + Qoe 
Thus the measure d(v@) is given by 
(1. 12) d(va) | det a |?- | Ve1042 + dv. 


In the general case, the measure d(v@) is obtained as follows: Let yw be the 
character of the diagonal sub-group of GZ given by 


2(n-2j+1) 


(1.13) |e 
For each decomposable a in GL, let 
=p (Cc) 


where c is the diagonal part of a in the decomposition (1.2). Then with a 
fixed, n(va) as a function of v is defined on all but a set of measure 0 in V; 
and 

(1.14) d(va) = p(va)dv. 


This is proved in [3] for the case in which a has determinpant 1. The 
general case may be deduced from this with the aid of the following observa- 
tions. (i) If a€ GL and b is a non-zero scalar multiple of a then a and b 
determine the same transformation of V; furthermore, (1.6) and (1.7) 
show that the digaonal parts of va and vb differ by a multiplicative factor. 
(ii) ifa€ GL then (deta)-/"a€ G. (iii) In (1.13) the sum of the exponents 
is equal to 0. 

If A is any character of the diagonal sub-group of GL its domain of 
definition may be extended as above. Thus if a—vucv with we U, ve V, and 
ce diagonal we shall put 


(1.15) A(a) =A(Cc). 


After this extension A is no longer multiplicative but has the following 
properties. If c is diagonal and a is decomposable, then ca and ac are 
decomposable, and 


(1.16) A(ca) =A(ac) —A(c)A(a). 
If v€ V then A(v) —1, and 
(1.17) A(vab) =A(va)A[ (va) 


whenever va and vab are both decomposable. 
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We shall say that a character of C is unitary if it maps C’ into the 
multiplicative group of complex numbers of absolute value 1. To each con- 
tinuous unitary character A of C is associated a unitary representation 
a— T(a,r) of G on the Hilbert space, L.(V) of square integrable functions 
on V. These representations are known as the principal series and are defined 
for f in L.(V) by 
(1.18) T (a, )f(v) =A(va) p3 (va) f (va). 


2. The decomposition V — ZZ’. In this section we decompose V into 
the product of two sub-groups Z and Z’ and study the action of G on V in 
terms of this decomposition. 

Let and m be integers such that and m—Il<n—1. 
For each a€ G, let a be the n X n matrix specified by the following conditions. 


(i) = Aj, [=j,kSm. 
(ii) k<l or k>m. 
(iii) dj,—=0O in the cases not covered by (i) or (ii). 


We shall denote by @ the submatrix of a formed from the entries aj, 1S j, 
k=m. The mapping v->%, ve V is a homomorphism of V onto a sub- 
group Z=Z(l,m); Z is contained in V and is isomorphic to V(m—1-+ 1) 
via the mapping zz. Let Z’—Z(l,m)’ be the kernel of the homomorphism 
v—v. Then Xn identity matrix is the only element of V which is common 
to both Z and to Z’. Moreover, an easy computation shows that iv € Z’ 
for each v in V. Thus to each v€ V there correspond unique elements z and 
z such that 

(2.1) 2€Z, 


To illustrate we consider the case n—=4. A typical element of 7(2,4) has 
the form 


1 0) 0 O 
0) 0 
0 Uso 1 0 


LO Uss 1 


whereas a typical element of Z(2,3)’ has the form 


1 0 O 


Vs, 1 0 
V3, O i 


Var 1 


= 
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We shall denote by dz the invariant measure in Z obtained from the 
product of the differentials (1.8) 


jn = AX AY jx mi: 


Since Z’ is a closed normal sub-group of V, it follows ({9; p. 133]) that 2’ 
is unimodular and also ([9; p. 119]) that 


(2.2) dv = dz dz’ 


where dz’ is a suitably chosen invariant measure on Z’. We shall not need 
the explicit form of dz (actually, here the most obvious candidate for dz’ 
works). 

If bd is non-singular, the mapping z— zd is well defined and carries 
Z into Z. The transformed measure d(z5) is obtained as follows: Let jm be 
thet character of C defined by 


(2.3) Him =I | 


j=l 


2(m+1-2)) 


It then follows from (1.13) and (1.14) applied to the case n=m—I-+1 
that 
(2.4) d(2b) = prm(ab) dz. 


For certain a in G, 6=4d is non-singular, and the mapping z— 2a, z€Z 
coincides with the mapping z— 2). We shall be interested in two sub-groups 
of G whose elements have this property. One of the sub-groups is the diagonal 
sub-group C. For suppose c€ C. Then z¢ =c-zc for each z in Z, and since 


(c-12c) jx == 


it is evident that c and é induce the same mapping of Z into Z. The other 
sub-group we shall call U/+*=—U*(l,m). It contains U and is obtained as 
follows: Let 1 —X(l,m) be the sub-group of G consisting of all 2 such 
that The mapping X is an isomorphism between XY and 
G(m—I-+1). In addition, denote by Y = Y (l,m) the kernel of the homo- 
morphism u=wt, we U. It is then easily seen that YN Y = {1,}, that 1 
is in the normalizer of Y in G, and that the product, XY is a sub-group of G. 
We shall put Ut—=XY. To illustrate we again consider the case n—=4. A 
typical element of U*+(2,3) has the form 


Eu 
0 Use Urs Urs 
Ue Un Un 
0 0 


| 
| | 
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Let us now determine in general how an element of U* acts on Z. To do 
this we ask when an element xy of U* is decomposable. Since ryx*€ U and 
ry = axyx*-x it follows that zy is decomposable if and only if x is decom- 
posable ; furthermore, the elements c and v in the decomposition (1.2) of .ry 
depend only on z. On the other hand, since XY is isomorphic to G(m—!/-+- 1) 
it is evident that c and v are members of X and in fact that v€ Z. From these 
observations we obtain the following result. 


Lemma 1. Let 2€Z, x€X, and ye Y. Then zxy ts decomposable if 
and only if za is decomposable. If zx and zay are decomposable then 


(1) = € Z 
(2) diag (zry) = diag(zr) € X. 

3. The operators A(p,). In this section we shall use the decomposi- 
tions V—ZZ’ in defining a class of operators A(p,A) in L2(V). These 
operators play a crucial role in our work on the representations of G. In order 
to understand something about their group theoretic significance it will be 
necessary to introduce some terminology. 


Let & be an integer such that 1 =[k=n—1, and Z—=Z(k,k+1). Let 
p=px be the element of Y —Y(k,&-+ 1) which satisfies 


0 17, 
(3.1) | 
Suppose in addition that - -,m, are integers, that s,,52,- are 
complex numbers, and that A is the character of C given by 


n CG; my 
| | 
We shall now obtain an explicit expression for the function A(zp), z€ Z. 


If z€ Z then 
1 off 
1 


Hence the mapping z— Zx,:; is an isomorphism between Z and the additive 
group of complex numbers; to simplify the notation we shall also denote 
Zksizr by z We use this convention in the statement of the following lemma. 


‘For further clarification of the role of the matrices p, see the discussion at the 
beginning of Section 4 and in Section 6 following equation (6.4). 
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LEMMA 2. Suppose z€ Z, u€ U, and that d is defined by (3.2). Then 
for 


A(zup) =A(zp) = 


Proof. Since Ut==XY we may write re X, ye Y (actually 
v=u). Thus Because Y is normal in U*, 
poyp€ Y, and it follows from Lemma 1 that zup is decomposable if and 
only if zap is decomposable. 

In addition we have 

Lo 


Now if we also write x for Uzx.1 it follows that 
Hence if 
1 — — 0 1 
=| E 


= 
0 1 
—z 


0 1 za + 
a L L 


From this we see that zwp and zp are decomposable if and only if 2 = Zr1xn A035 
if this is so, diag(zup) = diag (zp) X, 


1 
_% 0 


and by (1.15) 


Now let » be the character of C defined by (1.13). Applying Lemma 2 
we obtain 


8, 


The operators A(p,A) are defined in L.(V) at least formally by the 
integral 
1 


(3.4) A(p,r)f(v) A* (zp) (zp) f (270) dz. 
y(A) Jz 


| 
11 
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The factor y(A) =y(mi— Where for any integer m and 
complex number s 

(3.5) y(m, 8) = Q8 


r(! m | 


To study the operators A(p,’) we introduce another class of operators. 
If m is an integer and s is a complex number, 0 < Re(s) <1, we define an 
operator A(p,m,s) in L.(V) by 


(3.6) A(p,m,s)f(0) f : 


As we shall show, the integral in (3.6) is well defined and A(p, m,s)f € L2(V) 
for a dense set of functions f in Z.(V). Thus (3.4) will define an operator 
in L.(V) whenever 0 < Re(s,— 8x41) <1. For the domain D of the operators 
A(p,m,s) we shall take all bounded Baire functions with compact support. 


(2-1y) dz. 


Lemma 3. Let m be an integer, s a complex number, 0 < Re(s) <1, 
and f an element of D. Then 


(1) (8.6) converges absolutely for all ve V; 

(2) A(p,m,s)fé€ L2(V) ; 

(3) the mapping s>A(p,m,s)f has a unique extension to Re(s) =0, 
so that as a function of s with values in L(V) it is continuous in 
05 Re(s) <1; 

(4) if Re(s) =0, the operators A(p,m,s) are isometric and extend 
(uniquely) to unitary operators on L.(V) ; 

(5) if Re(s,) = Re(s.) =0 then 

A(p, m1, A (p, Ms, 82) = A(p, my + ms, + 
for all integers m, and m.; 
(6) <A(p,0,0) =I =the identity. 


Proof. (1) If f is a function on V, it follows from (2.1) that we may 
write f(v) =f(zz’) =f(z,2’). In addition, if f is integrable (2.2) shows 
that 

Sf(v)dvu = f f(z, 2) dz dz’. 


Thus if we identitfy z with its entry 2;,,, and use the fact that Z is isomorphic 
to the additive group of complex numbers, we may rewrite (3.6) as 


/ 
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d In this integral w is a single complex variable, and dw is the 2-dimensional 
element of area. If we take absolute values in (3.7) and disregard the 
multiplicative constant, we have 


| A(p,m, s)f(z,2’)| Sf | w f(z + w, 2) | dw 
where 8) = Re(s), 0<8) <1. Therefore 


| A(p, m, s) f(z, 2’) | 


We shall denote the first integral by g:(z,2’) and the second by g2(z,2’). 
) Since 0 < s) <1, | w |-?** is integrable over the unit disc and square integrable 
over the set where |w|>1. Because | f(w,2’)| is bounded and has compact 
support as a function of w, it therefore follows that the integrals defining 
9:5 Jo exist for all z, 2’. This proves (1). 


| (2) Let K be a compact set containing the support of f(z,2’), K, the 
projection of K on Z, and K, the projection of K on Z’. Then K,, Kz are 
both compact and have finite measures | K,|, | K.|. Denoting lub |f| by M 
and using Young’s inequality, we obtain 


f| 91 (2, 2’) dz= | K, w |-?+8 dw)? <0 


and 


MP | K, f | w dw 
7 |wl|>1 


If g(z, 2’) =A(p,m,s)f(z,2’), it follows that 
f|g(z,2)|? dz S Constant - M?(| K,| + | K2|?). 
Note that f(z, 2’) =0 if 2’¢ K.. Hence g(z,2’) =0 when 2’¢ K,. Therefore 
+ | <0 
which proves (2). 


2 dzdz’ = Constant - M?(| K, 


(3) Let ¢ be a complex variable. Then the Fourier transform of 
f(z, 2’) with respect to the z variable is the function f(t, 2’) given by 


(3.8) R(t, 2’) = fet Rett) F(z, 2’) dz. 
2r 


Let us put H.(w) ¢€=0. We need the 
y(m, 8) | w | 


736 R. A. KUNZE AND E. M. STEIN. 


following fact (see Bochner [1], p. 39). If t40 


lin H. »-i Re(tw) 
lim f (w)e w= 


We shall also put g-(z,2’) = fH.(w)f(z—w,2)dw for «20. Then 
A (p,m, s)f (2,2) = 9o(z,2), and the argument used in (1) and (2) shows 
that g., « > 0, also belongs to Z.(V). Applying the dominated convergence 
theorem, we see that g-(z,2’) > go(z,2’) for each z,2. For «20, is 
dominated by the Z.(V) function 
wy 
h(2z, 2’) = | f(z—w, 2’) | dw. 

A second application of domianted convergence shows that ge— go in 
L.(V). We now denote by 2’) the Fourier transform (3.8) of 9-(z,2’), 
«e=0. By the Plancherel theorem, we see that ¢,.(t,2’) > Go(t,2’) in the 
L.(t,2’) norm. On the other hand, 


Since 27° t|-*, £40, we find that 


It follows from the Plancherel theorem that we may use (3.9) to extend the 
definition of A(p,m,s)f to all s in the strip 0S Re(s) <1. To prove the 
continuity of A(p,m,s)f as a function of s with values in L.(V) it is then 
sufficient to show that 


t m 
f(t, 2’) |? | |? dé de’ 0 


as s—>s,, 0S Re(s) <1, OS Re(s,) <1. We recall that f is bounded and 
vanishes outside the compact set K; hence, /(t,2’) is bounded and vanishes 
when z’¢ K,. Thus the above expression is bounded by 


Constant f | | |-# |? dt dz’ 
K, |t|S1 
+f 
|t|S1 


An easy application of the Lebesgue dominated convergence theorem shows 
that both integrals tend to 0 as s> 8}. 
It is obvious that the mapping s— A(p, m,s)f has at most one continuous 


| 
\ 
‘ 
\ 
| 
\ 
| 
( 
1 
i 
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extension to the strip 0 Re(s) <1, and the proofs of (4), (5) and (6) 
follows immediately from (3.9) and Plancherel’s theorem. This concludes 
the proof of Lemma 3. 


4, The intertwining properties of the operators A(p,). Let S be 
the sub-group of @ which is generated by the unitary matrices p= px, 
1<k=sn—1. Then S is contained in the normalizer of C, and there is 
little difficulty in showing that S is finite. For the purposes of this paper it 
will be convenient to refer to S as the Weyl group.°® 

If A is a character of C and the mapping c—>A(q?cp), c€ C is 
again a character. We denote this character by gd. Let us now suppose 
a—T(a,r) and a—T(a,r,) are two members of the principal series. Then 
by a fundamental theorem of Gelfand and Neumark [3] these representa- 
tions are unitarily equivalent if and only if there is an element q of S such 
that A, = qa. 

Their proof of this result is in part abstract, and in part quite concrete ; 
however, it yields only the existence of the intertwining operators.* In this 
section we shall exhibit specific intertwining operators between the repre- 
sentations a—>T(a,rA) and a>T(a, pA), p=pr,, 1LSkSn—1. 

Let 2 be the character of C defined by 


m C; mz 
(4.1) A(c) =I] | 
where m,,Mzs,° *,m, are integers and are complex numbers 


such that 0= Re(s;) <4, 1 Sjsn. If p=p, 1 SkSn—1 we shall put 
A (p,r) =A (Pp, ME — Sk — 

It follows from Lemma 2 and (3.4) that A(p,A) is independent of the 
representation of A given by (4.1). One can also see this directly. For if 
we also have 

n c; 

ao) 

ja \| 
Where m’;,m’s,- -,m’, are integers and +, 8’, are complex numbers 
then there is an integer / and a complex number w such that m;= m’;+ 1 and 
Hence 


M;— Mj. = m’ ;— mM jx and 8;—- = 8 173%. 


* The Weyl group as it is usually defined may be identified with a quotient group 
of NS. 

*In recent note (Doklady 1960, vol. 131, pp. 496-499; also Soviet Mathematics 1960, 
vol. 1, no, 2, pp. 276-279) Gelfand and Graev have announced some results dealing with 
integrals of the type (3.4). 


t 
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When Re(s;— = 0, A(p,A) is unitary on L2(V) (Lemma 3) and 
otherwise A(p,A) is unbounded—with domain D consisting of all bounded 
Baire functions vanishing outside compact subsets of V. 

We recall that every continuous unitary character of C has the form 
(4.1) where -,8, are purely imaginary, i.e., Re(s;) =0, lSjSn. 


THEOREM 1. Let G be the nn complex unimodular group, A © con- 
tinuous unitary character of the diagonal sub-group, and a>T(a,A\ the 
corresponding member of the principal series. Then if p=p,, 1SkSn—1. 


(4. 2) A(p,A)T(a,A) =T (a, pA)A(p,d) 
for all a€ G. 


Roughly speaking, the main idea of the proof is to first establish (4. 2) 
for a class of non-unitary characters. The necessity for this arises from the 
fact that the operators A(p,A) are not given explicitly by (3.4) when A is 
unitary. The result is then obtained for unitary characters by a limiting 
argument. In the proof we shall need a series of lemmas. 

If p=p,, Sk Sn—1, let py be the character of C defined by 


(4.3) (C) = | Cx |? | 

Lemma 4. Let and Z=Z(k,k +1). Suppose 
z2€Z,anda=cu,c€C,ueU. Then 

(1) d(2d) —pp(2a) de; 

(2) wp (zu) = ; 

(3) pp (c) =pi(c)pA(p ep). 

Proof. (1) By definition, wp, is the character py4: given by (2.3). 
Thus if } 4d, it follows from (2.4) that d(zb) =p,(zb)dz. On the other 
hand, since b = ¢u, 2a = (2€)i—2b, Z. Hence, d(zi) =pp(2b)dz. We 
also have pp(2a) = pp(cczcu) =pp(C)pp(ctzcu). By (4.3), =pp(6), 
and Lemma 1 shows that = Thus 

Hp (20) = pp pp (ab). 


(2) Suppose zw is decomposable. Then by Lemma 1, diag(zu) =c 
belongs to X(k,é/+1). Hence cz,cx,,,—1, and from (1.13) we find that 
we) = | [2-29 | — | | Thus = p(c) 


= pp(C) = 
(3) It is easily verified that (p*cp),;=Cra, (plep)iu—=c;,. and 


(pcp);—=c; for j~k and jA~Ak+1. Thus 
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| Cr | Cras | | Cass | Ch 
=| cx |? | Cees |? 
= pp (C). 


This establishes (3) and concludes the proof of Lemma 4. 
Let a be a decomposable element of G and A a continuous character of C. 
In order to simplify the notation we shall set 


(4.4) K (a,r) =A(a)pA(a). 
If a is not decomposable, we put K(a,A) —0. It follows that 


(4.5) K (va, A) =A(va)p3(va), vE V, ae G 
and that 
(4.6) K (zp, (zp) (zp), 2€ Z. 


We recall that D is the collection of all bounded Baire functions vanishing 
outside compact subsets of V. For f in D, let 


(4.7) T (a, A)f(v) =K (va, (va). 


It will also be convenient to denote the character contragredient to a 
given character and \’. Thus 


(4.8) N’(c) =A(c*), cE C. 

In the next three lemmas we suppose the following: 

(i) p—p, 1SkSn—1. 

(ii) a€G, fe D, and ge D. 

(iii) A is given by (4.1) and 0 < Re(s;—sy41) <1. 

(iv) (pa)’. 

In addition, we shall denote the inner product 

ffi(v)fo(v) dv 

of two functions f,, in Lo(V) by 

Lemma 5. Suppose T(a,A)f€L2(V). Then the integral 


K (zva, K f[ (zv) dz 


erists for almost all v in V, A(p,rA)g € L2(V), and 
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J, K (zva, K (2*p, X*) f[ (2v) a] dz. 


Proof. By Lemma 3, A(p,A)g€ L2(V), and an easy computation shows 


that 

(4.9) K(z*p,r*) K(zp,d’) 
y(A) y(A). 

Thus 


A (p,A)g) 
K (va, a) f(vd) dv | K A*) dz. 
Z 
Now, K (va, r)f (va) K g(z1v) is a Baire function on V X Z and 
(4.10) J | K (va, A) f (va) | av | K (2*p,A*) (z1v)| dz 
4 Z 
Thus, by Fubini’s theorem 


(T(a,r)f, A(p,rA)g) = ff K (va, A) K (2-1 p, f (va) G dadz. 


VXZ 


Since the homeomorphism (v,z) — (zv,z) is measure preserving, 


(T(a,r)f, A(p,A)g) = ff K (zva, K f[ (2v) @] (v) dvdz. 
VxZ 
The stated result now follows from a second application of Fubini’s theorem. 
Lemma 6. Suppose T(a*,A,)g€ L2(V). Then the integral 
K (va, pa) K (2p, f [2 (va) 
Z 
exists for almost all v in V, and 


Proof. By Lemma 3, A(p,A)f€ L2(V). Thus 
(A (p, A)f, T(a", A1)g9) 


(p.A)f(v) K (va, (va) dv. 
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On the other hand, since d(v@) =p» (va) dv, 


(A (PsA) 9) fA (0a) KL (0d) g (0) av 
By (1.17), K[ (vd,A,|] (va) w4(va), and as Ay’ = pa we find that 


(A(p, )f, 


A (0A) K (0a, pd) do 


Lemma 7. Suppose both T(a,A)f and T(a,d,)g belong to L.(V). 
Then 
= (A(p, (a7, a1) g)- 


Proof. It is easily verified that 
(4. 11) K K*) (2p, At), 


By Lemmas 5 and 6, it is therefore sufficient to show that for almost all v 


(4.12) A) K (ap, A)f[(2v)a] dz = K(va, pd) K (2p, [z(va) | dz. 


In order to do this, we first observe that corresponding to a, there is a 
measurable subset V, of V such that (i) V—V, has measure 0, (ii) the 
integrals in (4.12) exist for v€ Vo, and (ili) va is decomposable for each 
v€ Vy. To each v€ V, there is a corresponding element 6 wc in UC such 
that va = b(vd). In addition, there is a measurable subset Z,,. of Z, depending 
on v and a, with the property that Z—Z,_ has measure 0 and ab is decom- 
posable for each z€ Z,4. Since the mapping z—2zb carries Z into Z and 
d(zb) =pp(zb)dz (Lemma 4), it follows that the second integral in (4.2) 
is equal to 


(4.13) K (va, pr) K[ (26) p, (a6) (vd) (2b) de. 


If z is a fixed element of Z,, there are matrices wu, and c, in U and C 
respectively such that zu = 1,¢,(z%). Since b= uc, 2b = zuc = c(zu)c 
=1,¢,¢(zb). Thus diag(zb)=c,c and zva=zb(vd) (2b) (va). 
It follows that zva is decomposable, that diag(zva) =c,c, and that (zv)a 
= (zh) (va). Hence upon comparing (4.13) with the first integral in (4.12) 
we see that it is sufficient to prove 


(4.14) K (zp, K (c,¢, A) = K (c, pA) pp K[ (2b) p, A). 
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Now from the definition (4.4) of K and the multiplicative properties 
(1.16), (1.17) of the functions A, p» it follows that 
K[ (26) p, ] = (zt) cp, 
K(c*,r*)K[ (ati) prep, 
K K (pep, K[ (zit) p, 
By Lemma 1, K (zp,A-!) = K (zup, A**), and on the other hand, 


K (zup, = K (zu, K[ (zt) p, 
= K(¢,,A*)K[ (2%) p, A*]. 


From this we obtain the relation 
K[ (2b) p, = K (c1, K (pep, A*) K K (zp, A*). 
Therefore (4.14) reduces to the condition 
K (¢1,X) K (1, A7*) pp (C177) 


(4. 15) 
= K K (¢, pr) op (c) K (pep, 0). 


The left side of (4.15) equals w(c,)up(¢c.-*), and this equals 1 by part (2) 
of Lemma 4. 

In the right side of (4.15), the product K(c, pA) K(p™'cp,aA*) 
= while K A) K = pi (ce) pi(c*). Thus the right 
side of (4.15) reduces to pp(c)y4(c)p3(p*cp), and this equals 1 by part 
(3) of Lemma 4. This finishes the proof of (4.12). 


Lemma 8. Let d» be a continuous unitary character of C and p=px, 
1SkSn—1. For 0<s<h and ceC, let r*(c) =| |*rA(c), and 
=| |*A(ptep). Then, corresponding to each a in G, there exist 
subsets D(a,r) and D(a", pr) of D with the following properties. 

(1) D(a,r) and D(a", pa) are dense in Lo(V). 

(2) For f in D(a,d) 

lim |) T(a,A)f —T(a,A°)f 

(3) Forg in D(a’, pa) 

lim || T pA)g —T r®))g = 0. 


Proof. Let f€ D. Then by (1.7) and the definition of T(a,A®)f, it 
follows that 


T (a, r)f(v) = [h(v) (a, d)f(v) 
[h(v) pd (va) f (va) 
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_ | Di (va) 
where h(v) = Pa . We also have 

T (a, f(v) = [hi(v) (a, pr) f(r) 
where h,(v) = 


Let E be the collection of all functions f in L,(V) with the property that 
ht: T(a,A)f belongs to L.(V). Since h is measurable and 7'(a,A) is unitary, 
FE is dense in L.(V). If f isin D and « > 0, standard measure theoretic argu- 
ments show the existence of a function f, in D such that || f—f, |l,<« and 
|fe(v)| S| f(v)| ace. Hence | and 
therefore f, belongs to ZH. Let D(a,A) DME. What we have just done 
shows D(a,) is dense in L,(V). 

If fe D(a,vA) and 0<s <4 then 


(4. 16) | h(v) | T(a,A)f(v)| S +1) 
for all v in V. In addition, 


(a, — lim | h(v) |* 4) 


since the right side of (4.16) is a function in L.(V), it follows from the 
Lebesgue dominated convergence theorem that 


| T(a,A)f—T (a, r®)f =. 


Now let D(a‘, pA) be the collection of all functions f in D such that 
hy-T (a, pr)f belongs to L.(V). Then the validity of (3) and the fact 
that D(a, pA) is dense in L,(V) are seen by exactly the same arguments 
as we have used above. 


Proof of Theorem 1. We suppose p= px, 1 = k= n—1, and that d is 
a continuous unitary character of C. Let A), A, D(a,A), and D(a, pa) 
be defined as in Lemma 8. Then in order to verify (4.2), it is sufficient 
to show that 


(A(p,A)T(a,d)f,g) = (T (a, pr)A(p, 9) 
for f€ D(a,rA) and g€ D(a*, pA). Now (5) and (6) of Lemma 3 tell us 
that the adjoint, A(p,A)*, of A(p,A) equals A(p,A). On the other hand, 


the adjoint of the unitary operator T'(a,pA) equals T(a*, pr). Thus, to 
prove the theorem, we need only show that 


(4.17) (T'(a,)f, A (p,A)g) = (A(p,A)f, T (a, pa)g) 
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for f € D(a, A) and g € D(a", pr). Let 0<s <4, f€ D(a,d), and g € D(a", pa). 
Then both T(a,A®))f and T(a*,A,)g belong to Z.(V). In addition, 
Ai) = [p-A®], ie, is the contragradient of p-A®. It follows from 


Lemma 7 that 
(4.18) (T (a, f, A(p,A)g) = (A(p,A)f, T (a, 9). 


As s tends to 0 through positive values, the left and right sides of (4.18) 
tend to the left and right sides respectively of (4.17) (Lemma 3 and Lemma 
8). This concludes the proof of Theorem 1. 


5. Commutation relations satisfied by the operators A(p,). Let 
and q—p;, Then by (4.2), A(g,A)T(a,A) 
= T(a,qgd)A(q,A) for all unitary characters A. It follows that 


A(p, qr)A(q,A)T (a, 4) = A(p, ga) T (a, ga) A (q,d) 
= T (a, pqa)A(p, Qa) A (9,4). 


Here we have used the fact that (pqg)A—p(qgaA). Thus A(p,qd)A(q,A) is 
an intertwining operator for the representations, a—> T(a, A) anda— T(a, pqa). 
Let us now suppose that 1 =[k=n—2 and that p= pr. It is easy 
to see that pgp—gqpq. Thus we can write down two intertwining operators 
for the representations corresponding to A and pqpaA, namely 


A (p, pqa)A (q, pA)A (p, A) 

and A(q, pgd)A(p, gA)A(q,A). We shall show that these apparently distinct 
operators are in fact identical. That they differ by at most a scalar factor of 
absolute value 1 is clear from the fact [3] that the representations of the 
principal series are irreducible. Our proof will not use irreducibility. We 
present it because it brings to light additional properties of the operators that 
will be used later on. 

The operator A(p,A) equals A(p, Where is given 
by (4.1) and Re(s;)=0, 17S. From the definition of g, it follows 
that A(p, ga) = A(p, mi. 8; — Similarly, 


A(g, A) — — and 
A (q, pd) == A(q, Mi — 
We also have 


A(p, gpr) =A (Pp, Misa — — and 


A(q, =A (q, ME — Sar). 


A 
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Using these relations we see that the equality of the above intertwining 
operators for all A is equivalent to the validity of 


A (p, m4, 81) A (gq, + Mo, 8; + 82)A (p, Me, 82) 
= A(q, Me, 82) A (p, my + me, 8; + 82) A (q, M1, 81) 


for all integers m,, m. and all complex numbers s,, s.—subject to the condi- 


(5.1) 


tions, Re(s,) =0, Re(s.) =0. If we replace m,, s; by —m,, —s, and use 
(5) of Lemma 3 we find that (5.1) is equivalent to 


A (p,— — 8,) A — My, — 8) A (q, mo, 82) A (p, Mo, 52) 
= A(q, m2, 8.) A (p, Mes, 82) A (p, m,, —8,)A(q,— m1, —8,). 
By (5) and (6) of Lemma 3 
A m1, — 81) = [A (q, A (p, m4, 81) 


Thus (5.2) is equivalent to the statement that the unitary operators 
A(q,m,s)A(p,m,s), Re(s) =0 form a commutative family ;* however, the 
operators A(q,m,s) and A(p,m,s) do not commute (when p = px, ¢ = rit). 


THEOREM 2. Let p=p, and q=—p;, 1Sk<jSn—1. Then the 
operators A(q,m,s)A(p,m,s), Re(s) =0, m an integer, form a commutative 
family of unitary operators on L(V), and A(q,m.s) commutes with A(p,m.s) 
for all m, s if and only if j—k>1. 


Proof. Suppose ;—k>1. Then it is easily seen that the matrices in 
Z(k,k +1) commute with those in Z(j,j7-+ 1) (see Section 2). From this 
observation and (3.6) it follows by a simple argument that A(p,m,s) com- 
mutes with A(q,m,s). 

We suppose now that }—=k-+ 1 and consider the decomposition V = ZZ’ 
where Z=Z(k,k +2). Then Z is isomorphic to V(3) and contains the 
subgroups Z(k,k +1) and Z(k+1,4+2). In addition dv = dzdz’, and if 
f is a function on V, we may write f(v) =f (zz) =f(z,2’). Iff is a bounded 
Baire function with compact support on V, and 0 < Re(s) <1, it therefore 
follows from (3.6) that 


(5.3a) A(q,m,s)f(z,27) = J ( yi ay f 2’) dzx 


and 


(5.3b) A(p,m,s)f(z,2) = f( +) | f (Zeer 22, 2 ) 


| 


*The various equivalent forms of (5.1) were pointed out to us by W. F. 
Stinespring. 
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In (5.3a) and (5.3b) the integration is extended over the sub-groups 
Z(k,k +1) and Z(k-+1,k-+ 2) respectively. Tus neither of our operators 
affects the second variable 2’, and it is therefore sufficient to prove the 
remainder of the theorem in the case n 3. Thus in what follows we shall 
consider the group V(3) consisting of all complex matrices of the form 


1 0 0 1 0 0 

V=/ 1 1 OO 
Vo1 V32 1 U3 V2 1 | 


We shall sometimes denote an element v of V (3) by(%, 2,03). If w 


= (W1, Wo, Wz) then 
(5.4) WU = (Wz + V4, We + Ve, Wz + V3 + 


and wt= (—w,,—w.,w,w,—w;). The sub-groups Z(1,2) and Z(2,3) 
are given as follows: Z(1, 2) consists of all elements of the form z, = (21, 0,0) 
while Z(2,3) is the set of all elem ts of the form (0, 22,0). Rewriting 
(5.3a) and (5.3b) with the aid of (5.4) we obtain 


-2+8 f( V1 — 2, Vo, V3) dz 
and 


(5. 5b) A(q, m, s)f(v) | dz. 


1 m 
may (757) z V.— 2, Vz — di. 


When Re(s) =0, A(p,m,s) and A(q,m.s) are defined as in Lemma 3. 


To analyze the products A(q,m,s)A(p,m,s), Re(s) =0, it will be 
convenient to introduce another unitary operator F, which is closely related 
to the Fourier-Plancherel transform on V(3)—studied by Godement in [4]. 
In defining F we first denote by £;, the unitary operator characterized by the 
condition 


(5. 6) E,f (1, Vo, V3 = Vo, Us); Us 0. 
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It follows easily from the classical Plancherel theorem that there is a unique 
unitary on such that 


1 
(5.7) Eof (V1, V2, Vs) = (2)? W2, Wz) 


whenever f€ L.(V) and f(v1,w2,ws) is integrable with respect to We, ws. 
We now put £,F,. 
LemMMA 9. Let Re(s) =0 and fe L.(V). Then 


U301 
| V301 | 


(5.8) EA(q, m, 8)f (V1, V2; Vs) -( ) | Ef (v4, V2, Vs). 

Proof. Consider first the case in which 0 < Re(s) < 1 and f is a bounded 
function with compact support. Then by Lemma 3, A(q,m,s)f€ L2(V). 
Let us put 

-e|z| m 
H(z) cr) «20 


7 y(m, s) 
and 
Je(V1, V2) V3) = Z, V3 — 2U,) dz. 


Then go = A(p, m,s)f, and ge— go in L2(V) ase—0. The argument showing 
this is similar to one given in the proof of Lemma 3. If «> 0 we have 


< fdvsdvs f | He(z)| | f(v1, | dz 
= f | He(2)| def | | 
= f | He(2)| fs, 2,0») | dvedv, 
< +o. 


Hence we may use (5.7%) to compute H.g,.. Interchanging the order of 
integration and changing variables in the resulting triple integral, we find 
that 
Boge(v) =f (U1, V2, Us) f He(z) dz, 
Thus by (5.6) 
Ege(v) = Ef(v) f dz, 


As 0 


| HA(q, m,s)f—E4g- |lze—>90 and 


» | 


m 


| | 


g 
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It should be noted that (5. 8) holds for all f in Z,.(V) whenever Re(s) = 0. 
It follows that 
EA (q, m, 8) f (01, V2, V3) = | vsvs Hf (v1, Vo, Us). 
| | 
We now let s—s,) where Re(so) =0. Since H is bounded, Lemma 3 shows 
that HA(q,m,s)f—> EA(q,m,s))f in L2(V). On the other hand, the right 
) | vsvs Ef (v1, V2, Vs), both 


side of the above equation tends to ( = 
| | 


pointwise and in the norm. This proves (5.8) when f is bounded and has 
compact support. The validity of (5.8), for all functions in L.(V’), now 
follows by routine arguments. 


Lemma 10. If Re(s) =0, A(p,m,s) commutes with E. 


Proof. Let f (v1, v2, Us) =g (v1) h (v2, vs) where g and h are bounded and 
have compact supports. Then from (5.5a) and Lemma 3 we see that 
A(p,m,s)f = (A(m,s)g)h, 0S Re(s) <1, where A(m,s) is the operator 
A(p,m,s) in the case n= 2; thus we have 


It now follows easily from (5.7) and (5.6) that FA (p, m,s)f =A(p, m, s) Ff, 
0= Re(s) <1. Since A(p,m,s) is bounded, for Re(s) =0 and finite sums 
of functions of the form gh are dense in L.(V), it follows that A (p, m,.s) 
commutes with whenever Re(s) = 0. 

We have already defined the operators A(m,s) on L.(v,). It will be 
convenient at this point to introduce another family of operators B(m,s). 
These are defined on L.(v,), when m is an integer and Re(s) —0, by 


-m 
| 


|-* f (v1). 


(5.10) B(m, 8) (v1) == (=) 


It should be noted that B(0,0) is the identity operator and that 


B(m,, B( ms, 82) = B(m, + ma, 8; + 82). 


Now let Re(s) 0 and suppose f€ Z.(V). Then upon replacing f by 
E-*A(p,m,s)f in (5.8) and using Lemma 10, we obtain the relation 


EA (q. m, s)A (p, m, s) Ef (0, V2, Vs) 


(5.11) 
|v |-*A(p, m,s)f(v1, 02, 0s). 


 < 
y(m,s) Z| 
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In (5.11) we next set f= gh where g€ L.(v,) and h€ L2(v2, v3). Rewriting 
the right side of (5.11), with the aid of (5.10), we find that 


HA (q,m,s)A(p, m, 8) E-*f (v1, V2, Vs) 
(5.12) 


| vs |-*(B(m, s)A(m, (v1) )h(v2, vs). 


Thus the operators A(q,m,s)A(p,m,s) form a commutative family if and 
only if the operators B(m,s)A(m,s) do. We shall finish the proof of 
Theorem 2 by showing that this is indeed the case. We should also remark 
that B(m,s) does not commute with A(m,s) if m0 or s0. 


Lemma 11. Let V=V(2). Then the unitary operators B(m,s)A(m, s) 
on L2(V), m an integer, and Re(s) =0, form a commutative family. 


Proof. For each complex number z0, let L, be the operator defined 
on Baire functions f by 
L,f(v) = f(z), ve V. 
Then and since {| f(z*v)|?dv, is bounded 
on L,(dv); on the other hand, L, is easily seen to be unitary on the space 
L.(|v|-*dv). A straightforward computation shows that 


(5. 13) L,B(m, s) = |z|*B(m,s). 


Next consider A(m,s)f where 0 < Re(s) <1 and f is a bounded function 
with compact support. A simple argument by change of variables shows 


-m 
L,A(m, | z|-*4(m, s)f. 
Using statement (3) of Lemma 3, we then obtain the relation 


Z 
) |z|-*A(m,s), Re(s) =0. 


(5.15) L;A (m, 8) 


Combining (5.13) and (5.15) we find that 
(5.16) L.B(m,s)A(m,s) = B(m,s)A(m,s)L,, 20. 


In order to finish the proof, it will suffice, in view of (5.16), to show that 
the operators L,, z>40 generate a maximal abelian (self-adjoint) algebra of 
bounded operators on L.(dv). We shall prove this with the aid of the iso- 
metric mapping, M of L.(dv) onto L.(| v |-? dv) which takes f(v) into vf(v). 
If z40, we have L,Mf(v) = Mf(z?v) =z of =z MLf(v). Thus 


12 
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L,M =2z*ML, or ML,M-1=2zL,. Hence the weakly closed algebra generated 
by the operators MZ,M- on L,(| v |-*dv) coincides with the algebra generated 
by the unitary operators L,; since | v |-*dv is a Haar measure on the multi- 
plicative group of complex numbers, it follows, by a well known result, that 
the translations L, generate a maximal abelian algebra on Lz(| v |-?dv). 


This concludes the proof of the lemma and the proof of Theorem 2 
as well. 


Corotuary. If m is an integer and Re(s) =0, let J(k,m,s), 1Sk 
= n—1, be the unitary operator on L2(V) gwen by 


(5.17) J (k,m,s8) =A(pn+s,m,s)° -A( puss, m,8)A (pr, mM, S). 


Then for fixed k, the operators J(k,m,s) obtained by varying m and s form 
a commutative family. 


Proof. Since J(n—1,m,s) =A(pni,m,s), the corollary in this case 
follows from Lemma 3. If > 2, J(n—2,m,s) =A(pni,m,8)A(pn-2,M, 8). 
This case is covered by the theorem. We may therefore suppose n > 3 and 
assume by induction on n—k that the corollary is true for 2k n—1. 
Let us put X,=—J(k,m,,8,) and mz, 82). We note that com- 
mutes with Y;, if and only if X¥,Y;°=—Y,71X,. Now 


X,=X;A (pe, M1, 8,)A M1, 8;) 
and Y,—Y;A (po, me, 82)A(pi,m1,8,). Setting By, A(pj, 1S]j, 
k= 2, we find that 

X,Y, = Boo 


moreover, Thus = and 
and since B..-! commutes with 


Xi = X,Y 2" By. 


By induction, XY. commutes with Y.-1; hence Y,1X,. 


6. The normalized principal series. This section is devoted to the 
construction and study of a series of unitary representations of G which we 
shall call the normalized principal series. Before describing these represen- 
tations it will be convenient to make several notational conventions. 
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Let A be a continuous unitary character of the diagonal sub-group of G. 


Then we can write 
n Cc 


where are integers and *,S, are complex numbers 
such that Re(s;) 0, 1j<n. Moreover, we can choose m; and s; so that 


(6.2) 


and 


(6.3) 


Furthermore, this can be done in only one way; that is, (6.1), (6.2), (6.3) 
determine m; and s; uniquely. In what follows, we shall always assume these 
conditions are satisfied. Therefore, we may write = Mn381,°**5 Sn). 
Let us also put m,-+-- +--+ m,—r and 


(6.4) res (0,- - 0). 


We shall call the character resA the residue of A. The characters therefore 
split up into n distinct classes according to their residues. 

Let px, 1 =k S=n—1, be the unitary matrix given by (3.1). We recall 
that p1,P2,° * *;Pn+ generate a finite group S which we have referred to as 
the Weyl group. If we regard the elements of G as acting by right multi- 
plication on complex n-tuple space then p; sends (wW1,° Wx, * Wn) 
into (W1,° *,Wn). It follows that S is essentially the 
symmetric group of degree n. If 


then it is easily verified ‘that 


where p,A(c) =A(pxtep;,), c€ C. Hence if g is any element of S it follows 
that A and gad have the same residue. 

If A= (m,,° +, Sn) denote by J(k, A) the operator J(k, mx, 
defined by (5.17), and set 


(6.5) W(A) =J(1,A)J(2,A)- -J(n—1,A). 


Furthermore, if a—T(a,r) is the representation of the principal series 
corresponding to A, let 


(6.6) R(a,r) = W(A)T (a, A) W(A)7, G. 


n 
0= dm <n 
jal 
|| 
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Since W(A) is a unitary operator, (6.6) defines a continuous unitary repre- 
sentation a—> R(a,r) of Gon L.(V). These new representations a—> f(a, A) 
are, by construction, unitarily equivalent to the corresponding representations 
a—T(a,r) of the principal series. We shall call this new series of repre- 


sentations the aurmalized principal series. 


The following theorem which describes the behaviour of R(a.A) as a 
function of A is one of the main results of this paper. 


THEOREM 3. Let G be the complex unimodular group of degree n and 
A the collection of continuous unitary characters of the diagonal sub-group C. 
Denote by G, the sub-group of G which consists of all those elements a€ G 
with the property that am =0, 1SjSn—1. If a€ Gand rE A, let R(a,X) 
be the operator of the normalized principal series defined by (6.6). 


(1) Then for each fixed a€ Go, R(a,r) as a function of x depends only 
upon the residue of A; im fact 


(6.7) R(a,rA) = R(a, resX) = T (a, res d) 


where T(a,resr) is the operator (1.18) of the principal series, corresponding 
to the element a€ G, and to the character resi, given by (6.4). 


(2) In addition, 
(6.8) R(a,r) = R(a, ga) 
for all a€ G, AE A, and q in the Weyl group S. 


Remarks. (i) If we allow the elements of G to act as before on complex 
n-tuple space, then G, is the largest sub-group of @ leaving invariant the 
m—1 dimensional subspace of vectors for which w, 
Moreover, in the proof we shall show that a given element of G is either in 
Go or in Gopyn+Go. The study of the dependence of R(a,A) on d for a general 
element a€ G is therefore reduced by (1) to the study of the operators 
R(pn+,). In particular, the problem of the analytic continuation of the 
operators A(a,A) to non-unitary characters is essentially reduced to the 
problem of the continuation of the operators R(pn-1,d). 


(ii) Given part (1) of the theorem, part (2) follows from the result 
of Gelfand and Neumark [3] that the representations of the principal series 
are irreducible when restricted to G). For the representations of the nor- 
malized series are then also irreducible on G, and moreover, the represen- 
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tations corersponding to A and gA are unitarily equivalent. Thus there is a 
unitary operator B(A) such that 
B(A)R(a,A) = ga) B(A), G. 
since A and gd have the same residue, R(a,A) R(a, gard), a€ Go. Hence 
B(A)R(a,A) = R(a,A)B(A), Go. 


Because the representations are irreducible on G, it follows that B(A) is a 
non-zero scalar multiple of the identity. Thus R(a,A) =R(a,gA) for all 
G. 

On the other hand, it is not difficult to prove (2) directly, and we shall 
give a proof which does not use the irreducibility of the representations of Go. 


LemMMA 12. Let S be the group generated by py, and H 
the lower triangular sub-group, i.e., the set of all a€ G such that ayx=0 
forj<k. Then G is a finite union of double cosets of the form HqH where 
q is an element of S. 


Proof. See Gelfand and Neumark [3]. 


LemMMA 13. (7) ts generated (algebraically) by H and py, post 


Proof. Let G,’ be the sub-group of G, consisting of all elements a such 
that = 1 and Then G,’ is isomorphic to G(n — 1) 
and contains p:,° * *.Pn-2. Moreover, it is easily seen that every element of 
G, can be expressed as the product of an element of G,’ with one in H. The 
lemma follows easily from these observations and the result of Lemma 12 


applied to G(n—1). 
LEMMA 14. G Go U Gopn-1Go. 


Proof. Let a€ G. Then by Lemma 12 there is an element qg€ S such 
thata€ HqH. Let us suppose that a is not a member of G,. Then Lemma 13 
shows that is not a member of the sub-group S’ generated by 
It follows that there exist elements q’, q” in S’ such that g=q/prig” and 
hence that @€ Gopn+G@o. This is most easily seen from the interpretation of 
of S as the ‘symmetric group’ together with the following fact about permu- 
tations: If + is a permutation of the integers 1,2.: - -.” which does not fix n 
then there exist permutations x’, 7” fixing n such that 7’ 
where (n—1,n) denotes the transposition interchanging n and n—1. 

If w is a non-zero complex number we shall find it convenient. in what 
follows, to adopt the notational device of writing |[w] for | é rs 

| w | 


7, 
} 
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Lemma 15. Let m be an integer, Re(s) =0, and1SkSn—1. Then 
for each Cand A 


Ck 


—-| T(c,rA)A(pe m,s). 
Ck+1 


Proof. Let « denote the identity character. Then since A(vc) =A(c) 
for each \ in A and v in V it follows that T(c,A) =A(c)T(c,«). Hence 
d(c)T (¢,A1) (c,d) for all A,A, in A. Using this relation and (4. 2) 
we find that 


1 
Tf A: = +, an easy computation shows 
Ai (¢) Ck+1 


In addition, A (px,A1) =A (px, Mess, Now choosing so 
that m = m, — my, and s = 8; — S;,, and substituting into (6.10) we obtain 


(6.9). 


Lemma 16. Let gq=—p;, and p—=p, If m is an 
integer and Re(s) =0, let M(q,m,s) be the operator on L.(V) given by 


(6. 11) M(q,m, 8) f(v) = [vjea | Vins |* f(r). 
In addition, let 
(6. 12) T m, 8) = M(q, m,8)T'(q,€) 


where T(q,«) is the operator of the principal series corresponding to q,e. 
Then 


(6.13)  A(p,m,s)T(p, m, s) =T(p,—m,—s)A(p, m, s); 

(6.14) A(p, m,s)T(q, m’, s’) = T(q, m’, s’)A(p, m, s) if |k—j| >1; 

(6.15) A(p,m,s)T(q, m’, s’) =T(q,m + m’,s+ s’)A(p,m,s) if |k—j | =1. 
Proof. Let X= (m4,° +, *,8n). Then (1.18) and Lemma 2 

show that 


(6.16) T =T (q, mj — — 


We also have the relation 
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Furthermore, 
T (p, pr) =T (p, — Mj, — 


Putting these observations together and using (4.2) we obtain (6.13). 

If |k—j| >1 then T(q,A) =T(q, pa) and (6.14) follows immediately 
from (4.2). 

In proving (6.15) we shall suppose j—k-+1, the argument being 
entirely similar in the case k—j-+1. If m, s, m’, and s’ are given we can 
always choose so that m= Mx, = — = — Muro, and 
= Then it is also true that m+ m’=—m,— and s+s 
= Moreover, 


T pr) = T(q, Mi — Sk — 
From these relations and (4.2) we obtain (6.15). 


Proof of Theorem 3. (1) Since R(a,a2,A) = R(a,,r)R(a2,A) for all 
@;, 42 in G and A€ A, the validity of (6.7) for a,, a2 in Go implies its validity 
for their product a,a.. Now HVC, and by Lemma 13, G, is generated 
by and H. Thus it suffices to prove (6.7) when a€ V, a€ C, 
and a= pj, 1S jSn—?2. 

Let us suppose first that a€ V. Then in (1.18) A(va)p3(va) =1 for 
allve V. Thus 7 (a,A,) =T(a,Az) for all Ay, A2 in A. Using this relation 
and (4.2) we find that the operators A(p,;,m,s) commute with T(a,A). 
The operators J(k,m,s) given by (5.17) therefore commute with T (a, A.) 
and hence by (6.5) we see that W(A) also commutes with T(a,A). Thus 
R(a,rA) =T(a,r) and we have established (6.7) in the case that a€ V. 

We now assume that a—ce€C. We also suppose Mn; 
51, * *,8,) and that conditions (6.2) and (6.3) are satisfied. Then rewriting 
(6.1) we find that 


n-1 C Bx 
where 


By 8, * + 
Now let us observe that the operator J(j, mj, s;), defined by (5.17), contains 
A (px, mj, 8;) as a factor if and only if 7 =k =n—1. Thus the factors of the 


form A(p;,m,s) in the product (6.5) are precisely those of the form 
A (px, mj, 8;), 1SjSk. It therefore follows from (6.9) that 


a) Wa) 2 = 


A(c) 
Hence W(A)T(c,A) W(A) = T(c, res A). 
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We shall now prove (6.7) for ap; where 1=jSn—2. Since 
1=jSn—2, we see from (6.16) that T(p;,resA) =T(p;,e). Thus it will 
suffice to show that 


(6.18) W(A)T'(pj, A) = T (pj, €) W(A). 
Let A == *, Then by (6.16), 
(pj, A) = T (pj, mj — Mur, 85 — ) 
and by (6.14) 
(6.19) J (k,A)T (pj, 4) =T (pj, (kA), 
From (6.15) and (6.13) we see that 
(6. 20) J +1,A)T (pj, A) = (pj, mj, (7 +1, A). 
Using (6.13), (6.15), and (6.14) we now find that 
(6.21) J (J, A) (pj, mj, 85) = T (pj, €) (p,d).- 
By similar computations with (6.13), (6.14), and (6.15) we also find that 
(6. 22) J(k,A)T (pj, €) =T (pj, (kA), <j. 
Finally, we observe that (6.18) is implied by equations (6.19) through 
(6. 22). 


(2) If (6.8) holds for q:, q2 in S, then using the fact that (q.q2)A 
= 4q:(q2A) for all A we see that it is also valid for q,g2. It therefore suffices 
to prove (6.8) for the generators, p, of S. Suppose then that k is fixed and 
let p=p,. Then R(a,rA) —R(a, pad) if and only if 


W(A)T (a, A) W = W(pa)T (a, pA) W (pr). 
This condition is equivalent, however, to the condition that 
W (pa)1W(A)T (a,rA) = T (a, pA) (a). 


To prove this it will suffice, in view of (4.2), to show that A(p,d) 
= W(pr)7W(A). We shall prove the equivalent condition 


(6. 23) W (pra) = W(A)A(p, 
Now A(px,A) =A (pry — Mess, — S11) and hence by Lemma 3 
A (Dis A (prs Ski) A (prs — Mk, —S;,) 


(6. 24) 
== A (Dr — Mk, —s,)A (prs Mi+15 + 
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Thus J(n—1,A)A (par, =JS (n—1, paid), and when 1S=jSn—2, 
J(j, 4) =J(j, It follows from (6.5) that (6.23) holds when k =n —1. 
Now suppose Then 


J (k + 1, A (prs Mk+15 Sk41) J (k, Mk+15 
It follows from (6.24) that 
J (k, A)T(k A)A (Drs = J (k, Mk, J (k, Mr+15 Ska) A (pr Mx — Sx)- 


Furthermore, the corollary to Theorem 2 shows that J(k, mx, s,) commutes 
with J(k, Mx.1, 8x41), and since J(k, mx, 8,)A (px, — Mx, — Sx) =I (hk + 1, mx, Sx) 


we see that 


(6.25) (pu, A)? =I (Ke, ped) J (+ 1, pad). 


In addition, A (p,,A)-? commutes with J(j,A) when k+1<jSn—1, and 
J(j,A) =J(j, prA) whenever |&/—j|>1. From these observations and 
(6.25) we conclude that W(p,A) = W(A)A (px, A)-. 


7. Lemmas on analytic continuation of certain operators. The results 
of this section as well as those of the next will be used in Section 9 to con- 
struct uniformly bounded representations of G. The main idea of this 
construction is to continue the operators R(a,A), of the normalized principal 
series, analytically in the parameters specifying A. 

We begin by recalling some standard definitions and results concerning 
analytic operator valued functions. Let H be a Hilbert space and s— P(s) 
a function defined in a domain D of the complsx plane whose values are 
bounded operators on H. Then P(-) is said to be analytic in D if s—> (P(s)f, 9) 
is analytic in D for all f, g in H. By a well known theorem of Hille ([7]; 
p. 538) such a function is continuous in the uniform operator norm and has a 
complex derivative, again in the uniform norm. From this it follows that 
if P(-) and Q(-) are analytic operator valued functions in D, then so is their 
product s— P(s)Q(s). 


Lemma 17. Let y(m,s) be defined by (3.5) ands a+b. Then there 
is a constant K independent of m and s such that 


| y(m, 


|m|+2 
2 


(7.1) 


Proof. Let c= and Then 


28 
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_T(c—z) 
To estimate the factor fics-4) 
[10; p. 151]) which implies that log P(w) — (w—4) log w + w is uniformly 
bounded in any sector of the form —z-+«Sargw7—e (for fixed « > 0) 
the logarithms being principal values. In this connection we also make use 


of the following easily verified estimates. 


we use Stirling’s formula (see Titchrnarsh 


(7.2) <| 2], 


From (7.2), (7.3), and Stirling’s formula we find that 
log (ec —z) —logT (c+ 
= — 2zlog | c—iy | + i(2¢e—1)arg(c—iy) + 22+ B(z) 


where B(z) is uniformly bounded in —}< Re(z) <4. At this point it is 
a straighforward matter to derive (7.1) from (7.4). 
In the next three lemmas we shall be concerned with the analytic con- 


(7. 4) 


tinuation of various combinations of the operators A(m,s) and B(m,s) 
treated in the last part of Section 5 (cf. equations (5.9), (5.10), and 
Lemma 11.) 


Lemma 18. Let m be ce fixed integer and L,=L,.(v) where v ts one 
complex variable. Then the operator A(m,s)-B(m,s) initially defined on 
L, for Re(s) =0 can be continued into the strip 0 Re(s) <1 in such a 
way that 


(1) for each f, g in Lz the function 
s— (A(m,s)B(m,s)f,g) 
is analytic in 0 < Re(s) <1 and continuous in 0S Re(s) <1. 
(2) | A(m, s)B(m,s) || SKa(1+|m| +] 
where s=a-+1b, 0SaSd <1, and ||-|| denotes the usual operator norm. 


Remark. Part of the interest of this result lies in the fact that the 
natural continuations of A(m,s) and B(m,s) are unbounded operators for 
0 < Re(s) <1, but nevertheless their product A(m,s)B(m,s) remains 
bounded. 


Proof. Let D denote the linear space of all bounded, compactly sup- 
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ported functions in ZL, which also vanish in a neighborhood of the origin. 
If f and g belong to D and Re(s) —0 let 


®(f, 9,8) (A(m, s)B(m, s)f; g)- 


Then since A(m,s)*—A(—m,§5) (see Lemma 3) it follows that 


®(f, 9,8) = (B(m, s)f, A(—™m, 5)g). 
Next we shall define ®(f,g,s) when 0 < Re(s) <1. To do this we first put 


(7.5) B(m,s)f(v) = v|-*f(v), 0S Re(s) <1. 


Then B(m,s)f€ L., and if f(v) =0 when | v| <8 it follows that | B(m,s)f | 
<=$"|f]|. Hence, by dominated convergence, s—> B(m,s)f is continuous in 
0 <= Re(s) <1 asa function with values in By Lemma 3, s— A (—m,5)g 
is also continuous as a function from 0S Re(s) <1 into Ly. Thus setting 


(7.6) ®(f, 9,8) = (B(m,s)f, A(—m,5)g), 0S Re(s) <1 


we see that ®(f,g,s) is continuous in 0= Re(s) <1. Furthermore, ®(f, 9, +) 
is uniformly bounded in any strip of the form 0S Re(s) Sd, 0<d<l. 
For the explicit form of the Fourier transform of A(—m,5)g, which is given 
in the proof of Lemma 3, shows that 


| A(—m,5)g 2S 


|A(—m,a)g s=a+wb,0Sacl. 
Thus in 0= Re(s) Sd 
(7.7) | S(f,9,8)| S3* lle 
The bound given in (7.7) is of course dependent on f and g. 
We now suppose 0 < Re(s) <1, and in order to simplify the notation 


we shall as before write [v] for Tel Then from (7.6), (7.5), (5.9), and 
the fact that y(— m,5) = (—1)™y(m,s) we find that 


y(m,s)®(f,9,8) = f[v}-™| o|-*f(v) dv f [z]™| 2 + 2) dz. 


In the integral with respect to z, we make the transformation z— vz—v and 


obtain 
(7.8)  y(m,s)®(f, 9,8) = ff(v) du f [z—1]™| z—1 |-***g(2v) dz. 
Thus 


| y(m, 8) ®(f.9,8)| F(v)| dof] 2—1 | g (20) | dz 


. 
= 
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where s=a+ib and0<a<l1. Now setting 
h(v) = f|2—1|***| g(2v) | dz 
and using Minkowski’s inequality for integrals we find that 


fl2z—1| | g (20) |? Ka’ |g | 


where Kq’ = f|z—1|-**|z|*dz and is uniformly bounded in any closed 
interval contained in (0,1). In fact K,’ is continuous as a function of a in 
the open interval (0,1). Thus 


dv 2—1 g(2v)| Ke If lle le 
and setting 


F(s) = ff(v)dv f[z—1]™|z—1 


we find that the integrals defining F' are absolutely convergent in 0 < Re(s) 
<1. Moreover, because of the boundedness of K,’, Fubini’s theorem shows 


that 
f F(s)dx=0 
A 


for every triangle A contained in the strip 0 < Re(s) <1. Thus F is analytic 
in 0 < Re(s) <1. Now in view of the analyticity of 1/y(m,s) and (7.8) 
it follows that ®(f,g,s) is analytic in 0< Re(s) <1. If Re(s) =0 and 
s is fixed, the definition of ® shows that 


0 < Re(s) <1 


> ®(f, 9,8) 


is a sesqui-linear form on D. Thus, since s>®(f,g,s) is analytic in 
0 < Re(s) <1 and continuous in 0= Re(s) <1, for fixed f,g in D, we 
see that 

f.g— ®(f,9,8) 


is also sesqui-linear when 0 < Re(s) <1. This follows because a function ( 
is identically 0 if it is analytic in 0 < Re(s) < 1, continuous in 0 = Re(s) < 1, 
and vanishes when Re(s) = 0. 

Next we shall obtain a bound for ®. First of all we see from Lemma 17 
and the estimates given above that 


(7.9) | ©(f,9,8)| le 
where s=a-+1b,0<a<1, and K, is independent of f and g; however, K, 


| = 
> 
4 
4 
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does not remain bounded as a0. In order to get around this difficulty 
we shall use a standard'Phragmén-Lindeléf argument. 

We fix d so that 0<d<1. Then in (7.9) we may assume without 
loss of generality that Kqa=1. Now consider the function ®, defined by 


+s) (I OS Re(s) <1. 
When Re(s) =0, the operators B(m,s) and A(—m,S5) are unitary. Hence 
from (7.6) we find that | 9 2, Re(s) =0. Therefore 
| @,(s)| 1, Re(s) =0. 
And from (7.9) it follows that 
| ®:(s)| 1, Re(s) —d. 


In addition, (7.7) shows that (f, g,s) is uniformly bounded in 0S Re(s) Sd; 
thus ®, is also uniformly bounded in 0=Re(s)=d. We conclude that 
|@,(s)| 1, 0S Re(s) Sd. This shows that (7.9) can be replaced by the 
stronger estimate 


| &(f,9,8)| SKa(i+|m|+ fell lle 


(7.10) 
s=a+, OSasSd<il. 


Since (7.10) holds for a dense class of functions f, g there is for each s in 
0 < Re(s) <1, a unique sesqui-linear form 


®:(f, 9,8) 


defined for all f,g in LZ. which extends 
fg 9,8) 


and satisfies (7.10). Because the uniform limit of continuous (analytic) 
functions is still continuous (analytic) the function 


s—> 9,8) 


is continuous in 0S Re(s) <1 and analytic in 0 < Re(s) <1, for each f,g 
in L,. Let A(m,s)B(m,s) denote the operator on L, which is defined 
by ®,(-,°,s). It is then apparent that the operator valued function 
s—> A(m,s)B(m,s) satisfies (1) and (2). This concludes the proof of the 
lemma. 

We now define another operator C(m,m’,s,s’) on Lz=L2(v), where v 
is one complex variable, by setting 


762 R. A. KUNZE AND E. M. STEIN. 


C(m, m’, s,s’) 
(7.11) 
= A(m,s)B(m-+ m’,s+s’) —B(m+ m’,s+s’/)A(m,s). 
Here we assume that Re(s) == Re(s’) —0 and that m and m’ are integers. 


Lemma 19. Let m, m’ and s be fixed. Then the operator C(m, m’, s, s’\, 
initially defined on L, for Re(s)=0, can be continued into the strip 
—1< Re(s) <1 im such a way that 


(1) for each f, g in L, the function 
s— (C(m, m’,s,s’)f,g) 


is analytic in —1 < Re(s) <1, and 


(2) | C(m,m’,s,s)|| | +] +] 
where ||-|| denotes the usual operator norm, s=a-+ib, s’=ib’, and 
—l<a<l. 


Proof. The proof is similar to part of the proof of Lemma 18, and we 
shall therefore omit some of the details. We begin with functions f, g both of 
which are bounded, vanish outside a bounded set, and are 0 in a neighborhood 
of the origin. For such functions f, g we put 


(f, 9,8) =(B(m + m’,s + 8’)f, A(— m, 5)g) 
(7. 12) 
— (A(m, s)f, B(— m, — m’, —5+ 
where 0 = Re(s) <1. Then 

®(f, s) (C'(m, m’, 8, g) 
when Re(s) 0. On the other hand when 0 < Re(s) < 1 we find, by rewriting 
the integrals, that 
y(m, s) 9; s) 

=f | (v) 9 (2) 


Now by means of the transformation v= zv, z—z we obtain the relation 


(7.13) y(m,s)®(f, 9,8) 
(zv) g(z)(Lv]-™™ | v —1) dvdz 


|1—0 


which holds when 0 < Re(s) <1. We shall show next that the integral in 
(7.13) converges absolutely whenever f, g belong to L, and —1 < Re(s) <1. 
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Let J, denote the integral over the set where | v—1| <4 and J, the integral 
over the complementary set. To estimate J, we use the inequality 


which is valid for |v—1|4, s—a+1b, ’=ib’, and —1<a<l. It 
follows from (7.14) that 


By Minkowski’s inequality for integrals, 


flelgle—1<a 
Now consider J,. Notice that 
and that 
Hence 
|b 
|v-1|2% 
1 (1+ |v + |v dude 


and ifa<1. These estimates show 
that the integral in (7.13) is bounded by 


le 


for —1<a<1 and all f,g in Z,. Furthermore, the above argument, in 
which we obtained the estimates for |7,| and |JZ.|, can easily be modified 
to show that the integral in (7.13) is analytic as a function of s in 
—1< Re(s) <1. From (7.12) it follows that @(f,g,s) is continuous in 
0S Re(s) <1. 


Hence we may use the integral in (7.13) to define 
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y(m,s)®(f,g,s) for —1< Re(s) <1. Then from the above estimates and 
Lemma 17 we conclude that 


(7.15) | ®(f,9,8)| lle lg le. 


for —1<a< 11 and all f, g in L,. Thus statements (1) and (2) of the 
lemma are satisfied by the operator C(m,m’,s,s’) which is defined by the 
form ®(-,-,s). 


Next we define another operator by setting 
(7.16)  E(m, m’, s,s’) = A(m, s)B(m + m’,s + s’)A(— m, —s)B(— m, —s). 
Here we assume that m and m’ are integers and that Re(s) == Re(s’) =0. 


LemMA 20. Let m, m’ and s’ be fixed. Then the operator E(m,m’,s,s’), 
initially defined on L, for Re(s) 0, can be continued into the strip 
—1< Re(s) <1 in such a way that 


(1) for each f, g in Lo, the function 
s—> (E(m, m’,s, s’)f, 9) 

ws analytic in —1 < Re(s) <1, and 

(2) | m’,s, 8’) S + | m| + | +] 
where s=a- ib, =ib’, and —1<a<. 

Proof. If Re(s) =0, (7.11) shows that 

A(m,s)B(m + m’,s +s’) =O(m,m’, s,s’) + B(m + m’,s +38’) A(m,s). 

Hence rewriting (7.16) we find that 


E(m, m’, s,s’) = C(m, m’, s, s’) A (— m, — s) B(— m, — s) + B(m’,s’). 
In view of the two preceding lemmas we may now extend the definition of 
E(m, m’, s,s’) by setting 


(7.17)  EH(m,m’,s, s’) =C(m, m’, s, s’)A(— m, — s)B(— m, — s) + B(m’, 
—1< Re(s) =0. 


On the other hand when Re(s) = 0, it is possible to rewrite (7.16) in another 
way. For it is easy to see that 


A(m,s)B(m-+ m’,s +8’) -C(—m,—m’,—s,—S’) 


= H(m,m’,s, s’)B(— m’, — s’) —I. 
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E(m, m’, s, s’) 
== A(m, s)B(m, s)- B(m’, s’)C(m, m’, s, s’)B(m’, s’) + B(m’, s’). 


By Lemmas 18 and 19, the right side of this equation is well defined for 
0< Re(s) <1. We shall put 


(7.18) E(m, m’, s,s’) 
= A(m, s)B(m, B(m’, s’)C(— m, — m’, —s, — 8’) B(m’, s’) + 8"), 
Re(s) <1. 
To prove (1) let us suppose f and g are fixed functions in Z.. Let 
&,(s) = (H(m, m’,s,s’)f,g), 0S Re(s) <1. 
Also let 
P(s) = A(m,s)B(m,s) and Q(s) = B(m’,s’)C(— m, — m’, — s, — s’)B(m’, s’). 


It follows from Lemmas 18 and 19 that P(s) and Q(s) are analytic in 
0 < Re(s) <1, and hence so is their product s—> P(s)Q(s). From (7.18) 
we conclude that®, is also analytic in 0 << Re(s) <1. Now suppose Re(s)) = 0. 
Then 


(P(s)Q(s)f,9) = (P(s)Q (50) + (O(s) f —Q (80) f, P(s) *9) 


and Lemma 18 shows that (P(s)Q(80)f,g) > (P(80)Q(80)f.g) a8 S— So. 
On the other hand, it also shows that || P(s)|| is uniformly bounded in every 
compact subset of 0 = Re(s) <1. Hence, since Q(s) is uniformly continuous 


|(Q(s)f —Q (so) f, P(s)*9)| SK Q(s) —Q(s0) If le lg 
as s—>&8. It follows that ®, is continuous in 0S Re(s) <1. Next we put 
b_(s) = (E(m, m’,s, s’)f,g), —1< Re(s) SO. 
Then, by arguments similar to those just given, it follows that ®_ is analytic 
in —1 < Re(s) < 0 and continuous in < Re(s)=0. Since #,(s) = &(s) 
when Re(s) =0, &, and &_ are analytic continuations of each other. This 


proves (1). Finally, we observe that (2) follows from (7.17), (7.18), the 
fact that B(m’,s’) is unitary, and the bounds given in Lemmas 18 and 19. 


8. Some lemmas on several complex variables. The first result we 
prove here is similar to well known theorems (cf. Bochner and Martin, Several 


13 
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Complex Variables) on the analytic continuation of functions of several com- 
plex variables, but deals with functions initially defined on a set which is 
considerably thinner than those which appear to have been treated in the 


literature. 


LeMMA 21. Let G(21,22,° + *,%m) be a gwen function of m variables 
(2; = 2; + ty;), imitially defined and continuous on the set where Re(z;) =0, 
j=1,2,---,m. Suppose that for every fixed j, and fixed 1Yj-1, 
1Y ° d 


as a function of 2; is analytically extendable into the strip | Re(z;)| <1 and 
satisfies |G@|<1 there. Then G is extendable into the tube 


(y’s arbitrary) 
is analytic there, as a function of the m variables, and satisfies | G| <1 there. 
Proof. Let G(ty1, =G(ty) and 
J 


It then follows easily from our assumptions that for each j 
(8.1) Sup | DG (iy)|Sn!, n=0,1,2,- 
Let H(z) = (2 (2—zm)~?. Then 
(k-+1)! 
ke 
D; H (ty) (2 — ty;)* H (iy). 
Now putting F(iy) = G(iy)H (iy), we find that 


k-0 


Hence by (8.1) and simple estimates 
(8. 2) | DF (iy)| S (n+1)!| H(ty)|. 


Let f(A) = fei F (iy) dy where <y,A> = + YmAm. Then 
integrating by parts, we find that 


= DinF (iy) dy. 


Thus in view of (8.2) and the fact that H is integrable, there is a constant 
A such that 


(8.3) Sup | aj*f(A)| SA(n+1)!, n=0,1,2,- -. 
N 
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It follows that / can be expressed in terms of f by means of the Fourier 
inversion formula. Thus 


Next we set 


F(x + ty) 


We shall show that this integral converges absolutely whenever x-+ ty is in 
the tube described above. Let 2 be fixed and such that =| 2|— pi. 


Then for each A, |<r,A>| Spmax|d;|. Thus 


Now using (8.3), we see that 


<K(n+m+2)!<o. 


Since the series 


(n+m+2)! | 
n! * 


is convergent, it follows that 
| f(A)| dA <a. 


Thus the integral in (8.4) is absolutely convergent, and the function F(z) 
= F (41, %m), Which it defines, is analytic in the tube | Re(z;)| <1. 
We now set 


F(z) 


G(z) = H(z) 


Dd | Re(z;)| <1. 

Then G is an analytic extension of our original function into the tube. To 
show that |@|=1 we suppose that there is some point z) such that 
|@(z.)| > 1. We then consider the function 


A 


G(iy) = G(iy) 


where w= and A is chosen so that Sup| G(iy)| 1. The function G 


y 
then satisfies our original conditions on G but obviously cannot be extended 
so as to be analytic at 2. Hence | G| <1. 


; 
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Lemma 22. Let T be a linear map of C” to C? such that T(R") = R:, 
and let T be the tube in C™ consisting of all z—=a-+ iy such that >| 2;| <1. 
Suppose G is an analytic function defined on T with the property that 
G(iy) = G(ty’) whenever Ty=Ty’. Then there is a unique analytic func- 
tion F defined on T(T) such that 


(8.5) F(Tz) =G(z), 


Proof. We may suppose the null space of T has dimension & where 
k>0. Then since T(R”)C T(R?) there is a linear map w>w of Ré 
onto the null space of T which sends real vectors into real vectors. Fix v¢€ R* 
and for a€T, let H(z) =G(z+w)—G(z). Then H is analytic in [ and 
H (iy) =0 for all y€ R”. Since T is a tube and is both open and connected, 
it follows by analytic continuation that 


(8.6) 
Now fix z€ T and let 
{we 0+2z€T}. 


Then I, is an open connected tube in R* which contains the set Re(w) =0. 
In [, we define a function K by setting K(w) = G(z+iw)—G(z). Then 
K is analytic in T, and by (8.6) K (iw) =0 for all ve R*. Thus, by analytic 
continuation, we see that G(z+ ww) —G(z) for all we€T,. In other words, 
G(z) = G(z’) whenever z and 2’ belong to T and Tz—Tz’. It follows that 
there is a unique well defined function F on T(T) which satisfies (8.5). 
Furthermore, our assumptions on 7 imply that T(T) is an open convex tube 
in C. Now let s’ be a fixed vector in 7(I). Then there exists a vector 2’ 
in T and a non-singular linear map L of C? into C” such that Ls’ = 2’ and 
TL is the identity on C?. If s is sufficientiy close to s’, F(s) =G(Ls). It 
follows that F is analytic in a neighborhood of s’ and hence that F is analytic 


in T(T). 
In our application we shall not use Lemma 21 directly but a slight 
variant of it. We shall be concerned with a function F'(s,, 8,° - -,S,) initially 


defined in a subset of the complex hyperplane s,-+s,+:--+s,—0. Our 
problem as before, will be to extend F so that it is analytic in a larger set. 
We shall say a function defined on a subset of s,+5,+---+5,—0 is 
analytic if it is analytic in the usual sense as a function of the n —1 complex 
variables *, 8-1 (with Before stating 
our result it will be convenient to introduce some notation. Let 0<d<1, 
8;=a;+%b; and B(d) denote the smallest convex set in the real hyper- 


wit 

ye 
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plane *+@,==0 which contains the points (a,—a,0,-- -,0), 
—d<a<_d, and all permutations of these points. 

Now let J(d) denote the tube in s,-+s,+- - -+ 8,0 whose basis is 
B(d), i.e., the set of all points (a, + + +,@n-++tbn) where (a4, d2,° *, Qn) 
€B(d) and b,+0.+-:-:-+6,—0, but otherwise the 6’s are arbitrary. 
Then J(d) is an open convex subset of s;+-s2.-+-- > -+5,—0, and with 
these conventions our result can be formulated in the following way. 


LemMaA 23. Let F be defined and continuous on the set where 
+> and a;=—0, j=—1,2,---,n. Moreover, suppose that 
for every pair j,k, Sj<kSn and fixed + (such that 
> b;=0) the function 


= F +, +, tbe +, thn) 


as a function of the one complex variable s =a -- ib has an analytic extension 
into the strip —d < Re(s) <d and | Fy(s)|S1 there. Then F has an 
analytic extension to the tube J(d) which satisfies the condition |F| <1. 


label the coordinates of points - 


Proof. Set m= 
in C” with two indices jk, 1S j <<kSn, and let T be the linear map of C™ 


onto the hyperplane s,-+-s,-+- - --+-s,—0 which is given by the equations 
k>j k<j 


Put G(iy) =F(iTy). Then it is easily seen that Lemma 22 applies to 
and hence that G can be extended analytically into the tube I whose basis 
is the set S| 2;| <1. Now let p—n—1 and T be the map T followed by 
the projection (81, 52,° +, —> (81, Applying Lemma 22 and 
the observation that 7(T) —J(d), we. easily obtain the conclusion of 
Lemma 24. 


9. Analytic continuation of the normalized principal series. In this 
section we consider the problem of the analytic continuation of the repre- 
sentations a— R(a,A) which are intially defined for unitary characters by 
(6.6). We shall obtain, in the end, a larger family of uniformly bounded 
representations, some of which are again unitary (the complementary series) 
and others which are not. 


Let J be the ‘tube’? J (1) which is described in the paragraph preceding 
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Lemma 23. We denote by A* the set of all characters A of the diagonal sub- 
group C of G having the form 


n 
C; 
jaa \| 
where are integers and 5, *,S, are complex numbers 
satisfying 


(i) 0S <n, 
(ii) (81, 8,° 
We shall use the notation 
(11, M2,* *, Mn 581, Sn) 
for the character given by (9.1). 


It is clear that A* contains the set of all continuous unitary characters 
of C, i.e., those which in addition to the above satisfy the condition Re(s;) = 0, 
j=1,2,---,n. As in Section 6, we shall denote the set of unitary charac- 
ters by A. 

A given complex or operator valued function 


F(A) = F(m, ma, Mx 5 81, Say ° *, Ss) 


defined on A* will be called analytic if for fixed m1, mo,- - -,m» it is complex 
analytic as a function of 81, 

Having made these conventions we are now in a position to state our 
main theorem. 


THEOREM 4. Let G be the complex unimodular group of degree n, C its 
diagonal sub-group, A the collection of continuous unitary characters of C, 
and A* the larger collection defined by (9.1). If a€ Gand X€ A, let R(a,d) 
be the operator of the normalized principal series given by (6.6). Then for 
each fixed a€ G, the operator valued function 


A— R(a, dA) 
initially defined on A can be extended into A* in such a way that 
(1) it is analytic in A*; 
(2) for each fixed X€ A* 
a—>R(a,r), a€ G 
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is a continuous uniformly bounded representation of G on L2(V), V being 


the group of lower triangular unipotent matrices in G; 


(3) if S is the Weyl group, the representations a—R(a,r) are 
invariant under S in the sense that 


R(a, gr) B(a, A) 
for all a€ G, XC A*, and q€ S, where ga is the element of A* given by 


(ga) (c) =A(qrcq), cE C. 


(4) if the contragredient of a character in A* or an operator on L(V) 
is denoted by a prime then 


R(a,0’) = R(a, 


for all a€ G and all X€ A*. 


Remarks. (i) In the proof of (2) we obtain an explicit but by no 
means best possible bound for the representation a— R(a,r). Our estimate 
may be described as follows: Let A= (m1, S15 S2,° * Sn) € A*. 
Then there is a real number d such that 0<d<1 and —d< Re(s;) <d, 
j=1,2,:--,n. We show that 


Sup R(a,A)| | H(a)| 


where ||-|| denotes the usual operator norm, Kq’ is independent of A, and 
H(X) is the analytic function on A* defined by (9.9). 


(ii) Let AE A* and a€G. Then A’(c) ==A(c"*), cE C and R(a,A)’ 
is the adjoint of R(a,A)-*. Thus A is unitary if and only if AA’, and in 
this case, statement (4) simply says that the representations of the normalized 
principal series are unitary. On the other hand, it is well known that G has 
another series of unitary representations, the so-called complementary series 
whose members are determined by certain non-unitary characters of C. 


In the present situation, we obtain the complementary series, more 
precisely representations unitarily equivalent to the complementary series, in 
a very simple fashion. In fact the following result is an immediate conse- 
quence of (3) and (4) above. 
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CoroLttaRy. Let X€ A* and suppose there 1s an element q€S such 
that = qr. Then a—R(a,d) is a unitary representation of G.® 


In the proof of the theorem we shall need a number of lemmas. 


Lemma 24. Let p=py+. where py. ts the element of the Weyl group 
defined by (3.1). For Re(s) =0 and 


let 
Then for fixed Xr, the operator Q(A,s) can be continued into the strip 
—1< Re(s) <1, in such a way that 
(1) for each f, g nm L.(V), the function 
(Q(,8)f,9) 
is analytic in —1< Re(s) <1, and 
(2) QQ,s)| (S| m;| +] 
where s=a-+ib, —1<a<l. 
Proof. By (6.6), 
R(p, A) = W(A)T(p, where W(A) J(n—1,A) 
and 
(9.3) =A (pass xy A (Pear, Mx, A (pry Mx, tx). 
Let us put 
W,(A) = J(1,A)- -J(n— 2, d), Ti(p, A) =I (n —1, A)T(p, — 1, 
and 


® That these representations are unitarily equivalent to the representations of the 
complementary series may be seen as follows: The two pairs of representations have the 
same characters as may be seen from formula (10.4) and the trace formula on page 115 
of [3]. Thus it follows by a theorem of Harish-Chandra in [6] that the representations 


are untarily equivalent. 
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Then R(p,r) = Wi(A)T1(p,A) Wi(A)-*, and it follows from (9.3) that 
W,(A )is independent of ib,-1; in fact, W1(A) = myo; 
Thus 
(9.5) Q(A, 8) = Wi(A)Q1(A, 8) Wi (A), Re(s) =0. 
Since W,(A) is independent of s and unitary, it will suffice to prove statements 
(1) and (2) of the lemma for Q, instead of Q. In doing this, we first note 


that 
T (p, A) =T (p, Mn+ — Mn, — thn) 


= M(p, — my, — bn) T (p, 
where M(p,— my, and T'(p, mn-1,tbn+.) are given by (6.11) and (6. 12) 
respectively. To simplify the notation we set 
Mn-1 = M, = b, m’ = — Mp, and = — — Dy. 
Then by the multiplicative properties of M 
T (p,rA) = M (p,m + m’, ib + 1b’) T (p, m, id). 


Thus 7',(p,A) = A(p, m, 1b)B(p, m + m’, ib + 1b’)T(p, m, ib)A(p, —m, —1b). 
Mcreover, (6.13) and (6.12) show that 


T (p, m, tb) A (p, — m, —1b) = A(p, — m, —1b)T (p, — m, — ib) 
= A(p,—m,—1b)h (p,— m, —ib)T (p,e€). 
Now setting 
(9.6) L(m, m’, ib, ib’) 
= L(m, m’, ib, ib’) 
= A(p, m, ib)M(p, m + m’, ib + 1b’)A(p, — m, — ib) M(p, — m, — 1b) 


it follows that T,(p,A) = L(m, m’, ib, ib’)T (p,e). Since b’ =—by1— Dn, 
l.e., since b’ = 6b, +--+ dn we see from (9.4) that 


(9.7) Q1(A, = L(m, m’, iby, + 8, 1b’)T (p,e), Re(s) =0. 


Let us consider the operator L(m, m’,ib,.,1b’). To study its action we 
use (as in the proof of Lemma 3, Section3) the decomposition 


L(V) 2), 


Z=Z(n—1,n). 
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We recall that this representation of Z,(V) arises from the expression of V 
as the semi-direct product ZZ’ and that Z is isomorphic to the additive group 
of complex numbers. If f¢€ L.(V) we can therefore write f(v) =f(z, 2’). 
Let us suppose that f(z,2’) =g(z)h(z’) where g€L.(Z) and he L.(2Z’). 
It is then easily verified that 


(9. 8) L(m, m’, tby-s, 1b’) f(z, 2’) = E(m, m’, thy-s, 1b’) g(z) h(2’) 


where E(m, m’, iby1,1b’) is the operator defined by (7.16). 


Hence upon applying Lemma 20, we see that (m, m’, ibn. s,1b’) can 
be continued so as to be analytic in —1 < Re(s) <1. In addition 


|| E(m, m’, +8, S + | m| + | +b] 4] 


where s=a-+ib and —1<a<l1. It now follows easily from (9.8) that 
L(m, m’, ibn, + 8, ib’) can be continued so as to be analytic in —1 < Re(s) <1 
with the same (operator) bound as H(m, m’,ib,.-+ s,1b’). In view of (9.7) 
and the fact that 7'(p,e«) is unitary the same can be said of Q,(A,s). To obtain 
the bound given in (2), we need only observe that | m | + | m’| =2(S| m; |) 
and that | b’| => |6;|. This concludes the proof of the lemma. 

Since we frequently need to refer to the following elementary result, we 


shall state it as a lemma. 


Lemma 25. Let F, and F, be analytic complex or operator valued func- 
tions defined on A*. Then Fi =F, if F,(A) =F,.(A) for all XE A. 


LemMMA 26. Let H(A) =H +, M351, be defined 
for XE A* by 
(9.9) AG) +2 (3 

j 

If 0<d<1 let A*(d) denote the set of all XC A* such that —d < Re(s;) < d, 
J=1,2,--+-,n. Then the operator valued function R(p,r) initially defined 
for X€ A can be continued into A* in such a way that 

(1) for each f, g in L2(V), the function 

(K(p,d)f,9) 

is analytic in A*, and 


(2) | R(p,A)|| S Ka | H(A)|, A€ A*(d) 


where Kq’ is independent of x. 


t 
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Proof. Let X= ++, A. Then if —1< Re(s) <1 
we shall put 
A; = (m,,° Mn 3 103, ° +s, tbn—S). 


It follows that A,€ A*. Now in view of (9.2) and Lemma 24 we may extend 
the domain of definition of R(p,A) by setting 


(9. 10) R(p, As) = Q(A, Ss); Re(s) <1. 


Next we define our operator for permutations of A;. To be specific, let us 
suppose ky,k2,- - -,kn is a permutation of 1,2,---,n. Then there is an 
element qg of the Weyl group, i.e., an element g of S such that 


By Theorem 3, R(p,A) =#(p, qd) for all A in A. We may therefore obtain 
an extension of our operator, which is invariant under S, by setting 


(9. 11) R(p,q- ds) =R(p, rs), —1 < Re(s) <1. 


From the bound given in Lemma 24 it is easy to see that || Q(d,s)|| 
<K,|H(aA)|. Thus since H is invariant under permutations it follows at 
once from (9.10) that 


R(p, q°As) || = K,|H(q:4s)| 
To simplify the notation, let A’ denote the set of all characters of the form 


gq: As, AE A, gE S. Then A’ can also be described as the set of all characters 
in A* of the form 


where 1 = j <k=n and —1 < Re(s) <1. At this point we have succeeded 
in defining R(p,A) for all characters A in A’. Furthermore we have the bound 


(9.13) | R(p,) || SKa| H()) 


where A has the form (9.12) and s=a-+dib. If f and g belong to L.(V) 
and A€ A’ we set 


(9.14) ®(f,g,A) = 


€ 
3 
4 
t 
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If 0<d<1 and Ki = Sup K, it follows from (9.13) that 
-d<a<d 


(9.15) | ®(f,9,A)|S Kad | H(a)| fllellg 


for all AE A’N A*(d), i.e., all A if the form (9.12) where —d < Re(s) < d. 
Next we fix f and g and show that ®(f,g,A), as a function of A, can be 
extended analytically into A*(d). In doing this, we may of course assume 
that || f We put 


®(f, 9, A) , * 


Then for fixed m,,- - -,m, and fixed },,b.,- - -,b, the function 


is analytic in —d < Re(s) <d. This follows from Lemma 24, (9.10), (9.11), 
(9.14), and the fact that 1/H is analytic in A*. Furthermore Fj; is bounded 
by 1. It now follows from Lemma 24 that F has an analytic extension into 
A*(d) which is also bounded by 1. Thus ®(f,g,\) has an analytic extension 
to A*(d) which satisfies (9.15). Moreover, since 
A*= |) A*(d) 
0<d<1 

our argument shows that ®(f,g,A) can be extended into A*. If A€ A is fixed, 
it follows from (9.14) that f,g—> ®(f,g,X) is a sesqui-linear form on L(V). 
By Lemma 25 the same is true for any fixed X€ A*. In addition 


| |A(A)| If llelg le 


for all A in A*(d). Therefore, @(-,-,) defines for each A€ A*, a bounded 
operator which we denote by R(p,A). It is then clear that R(p,2) satisfies 
statements (1) and (2) of the lemma. 


Proof of Theorem 4. (1) Let A= (my, +, Mn3 S15 80," Sn) € A*, 
and r=) 'm;. Then, extending the definition given in Section 6, we shall 
put 

resAX = (0,- - -,7;0,- - -,0) 


and call resA the residue of A. Suppose now that a€ Go, the sub-group of all 
elements a such that aj,—0, 1=j7=n—1. Then in view of Theorem 3, we 
may define R(a,A) in a consistent, in fact analytic, fashion by setting 


(9. 16) R(a,rA) = R(a, resd). 
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On the other hand if a€ G and is not a member of G, then, by Lemma 14, 
a€ GopG> where We can then write a= bpb’ with b and in G). 
By (9.16) and Lemma 26, the function 


A—> R(b,A)R(p, dA) A) 
is analytic in A*. In addition 
(9.17) R(a,A) = AE A. 
It follows from Lemma 25 and (9.17) that the function 
A— R(b,A)R(p, dA) A) 


depends only on a and not on the particular choice of b and 6’. It is therefore 
natural to extend the definition of R(a,A) by setting 


R(a,r) = R(b, A) R(p,rA)R(0’, A) 
for each AE A*, 


(2) Let a and 6 belong to G. Then by (1), R(a,A), R(b,A), and 
R(ab,A) are all analytic, as functions of A, in A*. Since the product of two 
analytic operator valued functions in again analytic, R(a,A)R(b,A) is analytic 
in A*, Furthermore, 


(9. 18) R(a,X)R(b, A) = R(ab, d) 


whenever A€ A. Applying Lemma 25 we see that (9.18) holds for all A€ A*. 
This shows a— R(a,A) is a representation. 

Next we show that when d is fixed the operators R(a,A) are uniformly 
bounded as functions of a€ G. If a€ Go, then R(a,Xr) is unitary and hence 
|| R(a,A)|| 1. On the other hand, if a¢€ G and is not in Go, a=bpb’, as 
above, with b, b’ in Go. It follows that || R(a,A) || = || R(p,A)||, and for the 
bound of R(p,A) we have the estimate given by Lemma 26. 

Now suppose f, g are fixed functions in L,(V) and that {a;,} is a sequence 
of elements in @ tending to the identity. Let us consider the sequence of 
functions 

Fy (A) = (R(ax,A)f,9), AE A*. 


We shall show that for each fixed A, there is a subsequence F;, such that 
F,,(A) > (f,g). From this it is easy to conclude that for each fixed A, 


| 
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a— (R(a,r)f,g) is continuous, and this being true for each f, g it will 
follow that a— R(a,r) is (weakly) continuous. If 


(m1, *, Mm; 81, 82,° € A* 


then we can find d, d’ so that 


(9.19) —d< Re(s;) << d, <In(sj)) << @, 
Let A(m,,ms,° * *,™m3d,d’) be the set of all characters in A* of the form 
A= (M4, Mo,* * 81, * Sn) 


which satisfy (9.19). Now fix m,,ms,---,mn, d and d’ and set A’ 
=A(m,,M2,: - *,mn3d,d’). Then from the bound given in Lemma 27, it 
follows that there is a constant K independent of k such that 


(9. 20) | F.(A)| SK. AE A’. 


Since the functions F;, are analytic in A’ and satisfy (9.20) they are equi- 
continuous there. Hence there is a subsequence F;,, which converges uniformly 
on A’ to a function F which is also analytic in A’. We shall show that 
F(A) = (f,g) for all A€ A’. To do this we make use of the known fact that 
the representations of the principal series are continuous. From this and 
(6.6) we see that a— R(a,A) is continuous whenever A is a unitary character, 
i.e., when A€ A. Hence F;,(A) > (f,g), if A€ A. In particular, this implies 
F(x) = (f.g) for all A€ AN A’. Thus by analytic continuation we see that 
F(A) = (f.g) for all AE A’. 


(3) Let a be fixed in G and q an element of S. Then R(a,qdA) is 
analytic as a function of A and so is R(a,A); in addition( these functions 
agree when AC A. By Lemma 25, R(a,gA) = R(a,A) for all AE A*. 


(4) Suppose a is a fixed element of G. Then to prove R(a, A’) = R(a, d)’ 
we must show that R(a,\’) = R(a"*,A)*. Let f,g€ L(V), and put F(A) 


= (R(a,A)g,f). Then F is analytic in A*, and if A= (m,, +, mn; 
F(A’) )= F (my, Mn3— 51, +, —5n). 


It follows easily that F(X’) is analytic, as a function of A, in A*. In addition 


(9. 21) F(X’) = (f, R(a,r’)g), AE A*. 


ll 
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On the other hand, if A is unitary then A=)’ and R(a,A) = R(a",d)*. 
Thus we have the relation 


(9. 22) (R(a*,A)f,g) = (f, R(a, ACA. 


Since both sides of (9.22) are analytic in A* (as functions of A), it follows 
from Lemma 25 that (9.22) holds for all A€ A*. This being true for every 
fand g in L.(V). we conclude that R(a*,A)* = R(a,d’), AE A*. 


10. Traces and equivalences. In the first theorem of this section we 
obtain a generalization of a trace formula due to Gelfand and Neumark, [3: 
p. 93]. With the aid of this formula we then determine the conditions under 
which two representations of our family 


a— R(a,A), X€ A* 


are equivalent. In particular we show that not all of our representations are 
equivalent to unitary representations. 

We begin by recalling some rather standard facts. If & is a Hilbert 
space and A is a bounded operator on &, the LZ, norm of A is defined by the 


equation 
(10.1)! || A = L= 
and || A ||, is the usual operator norm, || 4 ||. If || A |,1<oo then tr(A) 


= trace (A) exists, | tr(A)|< || A |]1, and A is said to be of trace class. On 
the other hand, if A and B are any two bounded operators on & then 
(10. 2) [AB |p lol Bly» 1S 


Next. let A€ A* and a€ G. If a has distinct characteristic values 


C1, * Cn let Cn] and 
10.3) 

= 


where D(a) —[[ | c;—c, |? and p ranges over S. If a does not have distinct 
ick 


characteristic values set y(a,A) =0. Then y(a,A) is a well defined measur- 
able function of a. 
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THEOREM 5. Let f be the convolution of two bounded functions with 
compact support on G. For each rX€ A*, let 


R(f,A)—= R(a,a)f(a)da. 
G 
Then R(f,A) is of trace class and 


(10. 4) y(a,A)f(a)da 
where y(a,2r) is defined by (10.3). 


Equation (10.4) is a generalization of the formula of Gelfand and 
Neumark [3: p. 93]. 


THEOREM 6. Let A, and dz belong to A*, and suppose A is a bounded 
operator on L.(V) with a bounded inverse such that 
R(a,A1) =AR(a, A.) A, a€ G. 
Then there is an element q in the Weyl group, S such that Ay=qAz, and 
then, by Theorem 4, R(a,A,) = R(a, Az), a€ G. 


On the other hand, suppose X€ A* and that 
a— AR(a,r)A" 


is a unitary representation, where again A is a bounded operator on L.(V) 
with a bounded inverse. Then a— R(a,d) is already a unitary representatin, 
and there is an element q in S such that X’ =qa. Thus if qa for all 
q€ S, a—>R(a,r) is not equivalent to a unitary representation. 


For the proofs of these theorems we require a series of lemmas. 


Lemma 27. Let U be a compact group and u->R(u) a continuous 
representation of U on a Hilbert space 8. Suppose 


sup || R(w) || <M. 
uéeU 


Then there exists a bounded invertible operator B on 9 such that 
(1) |BISM, |B+| SM, and 


(2) u—>BR(u)B" is a unitary representation of U. 
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Proof . Let & and suppose ||z || —1. Then for we U, 


0) SP, 


Hence if A = J ; R*(u)R(u) du and the measure of U is properly normalized 
U 
(Az, 2) M’*. 


If B is the positive square root of A then || B || = WV and | B* || SM as well. 
Part (2) now follows by the usual argument [2]. 


LemMa 28. Let f be a bounded function with compact support on G 


R(f,A) = J, f(a)R(a,r)da, A€ A* 
and 


Ky = sup || R(a,A) |. 


Then || R(f,A) where J; 1s a constant depending only on f and 
not on X. 


This is the key lemma.® 


Proof. Recall that R(a,A)|g, is unitary and irreducible.1° Hence 
R(a,r) is completely irreducible in the terminology of Godement [5]. Let 
U be the unitary sub-group of G, and w—D(u) a continuous irreducible 
representation of U. Let n(D) be the number of times that D occurs in the 
reduction of R(a,A)|v. Then since R(a,A) is completely irreducible it 
follows from a theorem of Godement [5: Theorem 2] that n(D) S degree (D). 
Moreover, by Lemma 27, there exists a bounded operator By such that 
| By || SK, || By* | S Ky, and 


P(a, BR (a, d) By 
is unitary when restricted to U. Since || P(a,A) || || By || || R(a@,A) || | Bx 


it follows that 
sup || P(a,A) || S K)%, 
aeG 


* The proof makes use of ideas occurring in paper [6] of Harish-Chandra. 
*°See (9.16), Theorem 3 and subsequent remarks. This is the first place where 
Wwe use the irreducibility of the principal series on Gp. 
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Because P(a,) is equivalent to R(a,A), n(D) is also the number of times 
D occurs in the reduction of P(a, 2) |v. 


Now let S be the closure of the support of f. Then S is compact and 
so is U-S8. Fixing A and setting P(a) P(a,r) we then have 


f(a)P(a)da— du lua) P (ua) da. 


From this it follows that 


fda f(wa)P(wa)du f(wa)P (ua) du da. 
Hence, since f f (ua) P (ua) du = f(ua)P(u)du)P(a) it follows from 
U U 
(10.2) that 


| Pre Ssup IP(a) | fuayP (uw) au 


We now estimate || f f(ua)P(w)du ||. Let u—>L(u) be the left regular 
U 


representation of U. Then D occurs exactly degree (D) times in L. Since 
n(D) & degree (D) it follows that 


In addition, the Peter-Weyl theorem shows that 
(u)du | f(ua) |? < sup | 
U U beG 
Hence because f has its support in S 


= || f |. meas (U-S) 


| 
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Combining these estimates we find that 
Py lle S 
However, R(f,A) = Thus 
|| R(f,A) S |] [leo Po lle Ba le S 


which proves the lemma. 


LemMa 29. Let f=fi*f. where f, and f, are bounded functions with 
compact support on G. Then for each XE A* 


f(a) R(a, A) da 
G 
is of trace class, || R(f,A) 7’, and 


tr R(f,A) 
is analytic in A*. 


Proof. Since R(f,A) =R(fi,A)R(f2,A). Hence || R(f, A) 
< || R(f1,A) |l2 | R(f2,A)l|2, by Schwarz’s inequality, which is valid in the 
present situation. The preceding lemma now shows that 


Thus tr R(f,A) is well defined. 


Let 91,92," °°, be a fixed orthonormal basis for L.(V), and let Py be 
the orthogonal projection on the subspace spanned by g1,92,° - °.9n- Then 
if A is any operator on L.(V) of trace class we know that 


tr(A) =limtr(P,AP,) 


n> 


and |tr(P,AP,)| || A Furthermore 


n 


(10.5) tr(PrAPn) => (Agus 


k=1 


Now let F(A) =tr(P»R(f,A)Pn) and F(A) =tr(R(f.A)). In view of 
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(10.5) and the fact that A> R(a,A) is analytic in A*, and hence that 
A\— R(f,A) is analytic there, it follows that F,, is analytic in A*. We also 
have the estimate 


| Fa(A)| S || R(f,A) /. 


Since K, is uniformly bounded over every compact subset of A*, the same 
is true of the sequence F,(A). As Fy(A) > F(A) for every A€ A*, it there- 
fore follows that F is analytic in A*. 


Lema 30. Let f be a bounded function with compact support on G 
and w(a,r) the function defined by (10.3). Then the function 


(10.6) H(a) = f(a)w(a,)da 
/G 
is analytic in A*. 


Proof. Recall that if A= (my, +, Mn3S1,S2,° Sn) then 


Furthermore 

and a is integrable and has compact support. The analyticity of H, i.e., 
the existence of the complex derivatives i +e. a follows by straight- 


forward differentiation under the integral sign, these operations being justified 
by the absolute convergence of the integrals in question. 


Proof of Theorem 5. If \ is a unitary character, then by its definition 
a—R(a,r) is unitarily equivalent to the corresponding representation 
a—T(a,r) of the principal series. Thus if f is the convolution of two 
bounded functions with compact support on G then 


tr R(f,A) =trT(f,A), AEA. 


On the other hand, Gelfand and Neumark have shown that 


tr7'(f,A) = J HQ). 


\ 


UNIFORMLY BOUNDED REPRESENTATIONS, II. 785 


But if F(A) =trR(f,A) then F is analytic in A*, by Lemma 29 while H is 
analytic in A*, by Lemma 30. Since F(A) = H(A) for A€ A, i.e., for unitary 
characters, it follows from Lemma 25 that F(A) = H(A) for all Xin A*. This 
finishes the proof of Theorem 5. 


Proof of Theorem 6. Suppose R(a,rA,) = AR(a,Az) A? where A is a 
bounded operator with a bounded inverse. Then if 


R(fA) f(a)R(a,r)da, LE A* 


it follows that, R(f,A,) —=AR(f,A,)A*. Now let f be the convolution of 
two bounded functions with compact support. Then R(f,A,) and R(f,A2) 
are both of trace class and by a standard property of the trace we see that 


tr(R(f,A1)) = tr(R(f,A2)). 
Thus by Theorem 5 


(10.7) J, a1) —¥(a, de) (a) da = 0. 


By a simple limiting argument (10.7) also holds whenever f is bounded and 
has bounded support. Hence y(a,A.) —y(a,A2) a.e. From this, (10.3) 
and the continuity of A, and A, we conclude that 


(pdr) (¢) (prz) (¢) 


for all diagonal matrices c in G. But since distinct characters are linearly 
independent, it follows that A; = qd» for some q€ VS. 

Now suppose a—T (a) is a unitary representation of G and that 
T (a) =AR(a,A)A-*. Then since T*(a) = T-1(a) =T (a), we see that 


= AR(a,r) 
or in other words, that 
R(a,r)*A*A = A*AR(a,d). 


But when a€ Go, R(a,A) is unitary and irreducible. This implies A*A = cl, 
where c>0. After proper normalization we may therefore assume A is a 
unitary operator. Hence a—R(a,2) is a unitary representation. It follows 
from Theorem 4 that R(a,A) = R(a,X’) where X’ is the contragredient of A. 
Thus by the first statement of the present theorem, \’ = qd for some q in VS. 
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IMAGES OF E, UNDER ACYCLIC MAPS.* 


By E. H. CoNNELL.' 


It has been shown in [8] that monotone mappings of F, onto EH, are 
compact. In this paper it will be shown that acyclic mappings of £, onto 
E, are compact. It is an open question whether or not monotone mappings 
of H; on EH; must be compact. It will then be shown that contractible open 
3-manifolds embedded in FE, but topologically distinct from H3, cannot be 
images of H; under a class of acyclic maps. It is unknown whether or not 
any contractible 3-manifold distinct from H; can be such an image. 


Notation. A map (continuous function) is monotone if each point- 
inverse is a compact connected set. A map is compact if the inverse of each 
compact set is compact. A map is acyclic if each point-inverse is a compact 
connected set with vanishing n-th homology groups for n —1,2,3- - - (Cech 
homology over a field G). If X is a space, (X +.) is the one point com- 
pactification of XY. If BC X, C(B) is the complement of B. If f: Y—Y 
and p€ Y, then f-*(p) C X will be called an orbit, and if A C X, the saturation 
of A is the union of all orbits intersecting A, i.e. saturation of A = f-"(f(A)). 
Manifolds are without boundary, i.e. a manifold is a connected separable 
metric space which is locally Euclidean. 


TuroreM 1, [f f is an acyclic map of E,(n >1) onto Y, a contractible 
n-manifold, then f is compact. 


Proof. Suppose for each p€ Y, J a neighborhood N of p such that 
f-*(N) is bounded. Then the inverse of each compact set would be closed and 
bounded, and the theorem would be trivial. Thus it may be supposed that 
dpé€ Y such that if O is an open set containing p, f-1(O) is unbounded. 
Let M=f-*(p) and let 8, be an (n—1)-sphere such that M is in the 
bounded complementary domain of S,. Let S. be the saturation of S, and 
note that S, like S8,, is compact, connected, does not intersect M and separates 
M from oo. (To see that S. is compact, first show it is bounded. Suppose 
Pn 18 a sequence in S, which approaches oo, and A, is the sequence of orbits 
With An. Let gn€ An MS, and note that some subsequence, also denoted 


* Received September 18, 1961. 
* This research supported by NSF Postdoctoral Fellowship (41057). 
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by gn, converges to g€ 8;. Let A be the orbit containing g and V be a bounded 
open set containing A. It is easy to show, using the fact that each A, is 
connected, that all but finitely many of the A,* are contained in V. This is 
a contradiction and shows that S, is bounded. A similar argument shows 
that S, is closed and thus compact.) Let O, be the bounded complementary 
domain of 8, which contains M and OQ, be the unbounded complementary 
domain of 8, Let S=H,—C(0O,U0.). Note that S is merely S, plus 
all bounded complementary domains of S, which do not contain M. Thus 
S is compact and connected and since it separates H, into exactly two pieces, 
H,1(S) =G. By the Vietoris mapping theorem ([6] and [1]), Hn+(f(S)) 
=G. Since Y is a contractible n-manifold, f(S) separates Y into two pieces, 
one unbounded (i.e. noncompact closure) and one bounded. Since f is onto, 
f(Q2) is the unbounded complementary domain and f(O,) the bounded one. 
Now the bounded one is an open set containing p whose inverse is the bounded 
set O,. This is a contradiction and proves the theorem. 

It should be noted that Whyburn’s theorem about F, is not obtained as a 
corollary, since he assumes only that inverses are compact and connected. 
However we do obtain the following: 


Corotuary 1. Jf f is an acyclic map of E, onto Y, a contractible n- 
manifold, and f is extended to f: +0)—>(¥ +) by =o, then 
f as extended 1s continuous. 


Proof. Let to. If f(an) +0 then J a subsequence, also denoted 
Ln, Such that f(z%,)-—> ye Y. Let N C Y be a compact neighborhood of y; 
then by Theorem 1, f-*(NV) is compact, but this is impossible. 


Corotuary 2. If f ts an acyclic map of E, onto a contractible n-manifold 
Y, the natural map from the decomposition space generated by f onto Y 1s a 


homeomorphism. 


Proof. By Corollary 1, f may be extended to the one point compactifica- 
tion. For compact spaces, this result is well known (p. 126 of [9]) and the 
desired result is obtained by removing the point at infinity. 

Note that each of the above corollaries implies, and each is thus equiva- 
lent to, Theorem 1. The following corollary is well known. 


CoroLuary 3. If f is a one-to-one mapping of FE, onto Ey, then f is a 


homeomorphism. 


Proof. The decomposition space is simply £, itself, and the result 
follows directly from Corollary 2. 
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R. H. Bing has posed the question: if a point-like decomposition of EF; 
yields a 3-manifold, must it be #;? Another question would be: if Y is a 
3-manifold and f maps F; onto Y such that point inverses are point-like, then 
must Y be #;? If a set is point-like, it is cellular (see [2]) and by the con- 
tinuity of Cech homology, must have the homology of a cell, i.e., must be 
acyclic. Thus by Corollary 2, these two questions are the same. At any rate, 
the question is partially answered here. The author thanks R. McMillan for 
his assistance. 


THEOREM 2. Suppose that Y ts a contractible 3-manifold with a locally- 
finite triangulation and that any finite subcomplex of Y can be semilinearly 
embedded in H;. Suppose f maps EF; onto Y with each f-*(p) a compact 
absolute retract (1.e. contractible and locally contractible). Then Y is homeo- 
morphic to Es. 


Proof. If Y is the monotonic union of closed 3-cells, each containing 
the previous in its interior, then Y is homeomorphic to #;. Thus it will suffice 
to show that for each compact subset NV of Y, 3 a 3-cell containing N. 

Suppose VY C Y is compact. By Theorem 1, f-1(N) is compact. Let 
M be the saturation of a closed 3-cell containing f-4(N). Let 0 be the 
unbounded complementary domain of J and note that 7.(0) 40. Since f is 
compact, Smale’s theorem (see [5]) applies and r.(f(0)) 40. Now by the 
sphere theorem ([4] and [%]), there exists a polyhedral 2-sphere S in f(0) 
which cannot be shrunk to a point in f(0). It plus its bounded complementary 
domain J(S}) can be embedded in EF; by hypothesis, and by Alexander’s theorem, 
S must bound a 3-cell; thus SUJ(S) is a 3-cell. Since S can be shrunk in 
SUI(S), I(S) Nf(M) and since M is connected, f,-/)C7(S) and 
therefore VN CI(S). Q.E.D. 

The Poincaré conjecture is equivalent to the following: if }° is a con 
tractible 3-manifold, then each compact subset of Y can be embedded in £ 
This is contained implicitly in [3]. R. McMillan and J. Kister have con- 
structed a contractible 3-manifold such that each compact subset can be 
embedded in Fs, but the manifold itself cannot be (yet unpublished). 


INSTITUTE FOR ADVANCED STUDY. 


E. H. CONNELL. 
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ON THE PREPARATION OF MANUSCRIPTS. 


The following instructions are suggested or dictated by the necessities of the technical pro- 
duction of the American Journal of Mathematics. Authors are urged to comply with these 
instructions, which have been prepared in their interests. 


Manuscripts not complying with the standards usually have to be returned to the authors 
for typographic explanation or revisions and the resulting delay often necessitates the defer- 
ment of the publication of the paper to a later issue of the Journal. 


Horizontal fraction signs should be avoided. Instead of them use either solidus signs / or 
negative exponents. 

Neither a solidus nor a negative exponent is needed in the symbols }, 1,1 .,1., which are 
available in regular size type. 

Binomial coefficients should be denoted by (," and not by parentheses. Correspondingly, 
for symbols of the type of a quadratic residue character the use of some non-vertical arrange- 
ment is usually imperative. 

Use the exponent } instead of the square root sign. 

Replace e‘ > by exp( ) if the expression in the parenthesis is complicated. 

By an appropriate choice of notations, avoid unnecessary displays. 

Simple formulae, such as A + iB = }0* or s,=a,+.-.--+a@,, should not be displayed 
(unless they need a formula number). 

Use ’ or d /da, possibly D, but preferably not a dot, in order to denote ordinary differen- 
tiation and, as far as possible, a subscript in order to denote partial differentiation (when the 
symbol @ cannot be avoided, it should be used as 9 /6x). 

Commas between indices are usually superfluous and should be avoided if possible. 

In a determinant use a notation which reduces it to the form det a,,. 

Subscripts and superscripts cannot be printed in the same vertical column, hence the 
manuscript should be clear on whether a,* or a*, is preferred. (Correspondingly, the limits of 
summation must not be typed after the =-sign, unless either 2:1" or 2": is desired.) If a letter 
carrying a subscript has a prime, indicate whether «@,’ or a’, is desired. 

Experience shows that a tilde or anything eise over a letter is very unsatisfactory. Such 
symbols often drop out of the type after proof-reading and, when they do not, they usually 
appear uneven in print. For these reasons we advise against their use. This advice applies 
also to a bar over a Greek or German letter (for the symbol of complex conjugation an asterisk 
is often allowed by the context). Type carrying bars over ordinary size italic letters of the 
Latin alphabet is available. 

Bars reaching over several letters should in any case be avoided (in particular, type 
lim sup and lim inf instead of lim with upper and lower bars). 

Repeated subscripts and superscripts should be used only when they cannot be avoided, 
since the index of the principal index usually appears about as large as the principal index. 
Bars and other devices over indices cannot be supplied. On the other hand, an asterisk or a 
prime (to be printed after the subscript) is possible on a subscript. The same holds true for 
superscripts. 

Distinguish carefully between 1. c. “oh,” cap. “oh” and zero. One way of distinguishing 
them is by underlining one or two of them in different colors and explaining the meaning of 
the colors. 

Distinguish between e (epsilon) and € or e (symbol), between w (eks) and X (multipli- 
cation sign), between 1. c. and cap. phi, between 1. c. and cap. psi, between l.c. k and kappa and 
between “ell” and “one” (for the latter, use t and I respectively). 

Avoid unnecessary footnotes. For instance, references can be incorporated into the text 
(parenthetically, when necessary) by quoting the number in the bibliographic list, which 
appears at the end of the paper. Thus: “ [3], pp. 261-266.” 

Except when informality in referring to papers or books is ealled for by the context, the 
following form is preferred: 

[3] O. K. Blank, “ Zur Theorie des Untermengenraumes der abstrakten Leermenge,” Bulletin 
de la Société Philharmonique de Zanzibar, vol. 26 (1891), pp. 242-270. 
In any case, the references should be precise, unambiguous and intelligible. 

Usually sections numbers and section titles are printed in bold face, the titles “ Theorem,” 
“Lemma ” and “ Corollary ” are in caps and small caps, “ Proof,” “ Remark ” and “ Definition” 
are in italics. This (or a corresponding preference) should be marked in the manuscript. 
Use a period, and not a colon, after the titles Theorem, Lemma, etc. 

German, script and bold face letters should be underlined in various colors and the meaning 
of the colors should be explained. The same device is needed for Greek letters if there is a 
chance of ambiguity. In general, mark all cap. Greek letters. 


All instructions and explanations for the printer can conveniently be collected on @ 
separate sheet, to be attached to the manuscript. 


In case of doubt, recent issues of the Journal may be consulted. 
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